Brightfind world usability day 2016 full deck final
Quantitative Meets Qualitative
Dr. Nichole Carelock
Manager User Insights, PenFed Credit
The Case of the Disappearing
The Problem: A blended learning math platform
for the State of Ohio was losing an alarming
amount of student data everyday.
Quantitative data told us we had 97%
(compulsory) engagement but only a 63% data
transfer from day to day.
Qualitative data told us our Log in page was
enticing ESL and non readers to sign in as Guest.
• Which dumped their data at the end of each session.
UX Methods are Categorized in
Quantitative methodology is defined as producing
numerical data which can be expressed as stats,
Qualitative methodology on the other hand, is
concerned with collecting textual (word) data
which is used to identify themes and patterns.
Measuring things Describing things
Numbers Natural description
Fixed state Negotiable state
Language of statistical analysis Natural language of participants
Formulation of facts Helps develop ideas
Closed questions in surveys Open questions in surveys
A/B Tests Interviews
Web analytics Voice of customers tools
Benefits and Drawbacks
Make data-driven business decisions Brings numerical data to life
Great at convincing to Stakeholders Great to plan or develop design strategy
Numbers require interpretation Time consuming- expensive
Easy to draw wrong conclusion Non-Automated
Free or Inexpensive tools Free or Inexpensive Tools
Facebook insights, Clicktale 4Q by iPerceptions
Remote, asynchronous focus grouping (google
Kiss insights, Optimizly Usertesting.com, GuerillaTesting (friends and family)
Quantitative Research → Qualitative
Helps explain why certain things happen after you
discover them through statistical analysis.
Qualitative Research → Quantitative
This order allows you to uncover trends and/or
preferences from individuals and then support
these claims on larger samples.
Quantitative and Qualitative
One simple way to obtain both
types of data is to send a
survey with closed and open
The closed surveys will give
you statistical results, while
open ended surveys provide
you with descriptive answers
The two methods work together
Sky and Water MC Escher
Using Mixed Methods to get
There are several steps involved in identifying the
right research method or combination of methods.
• Understand the problem.
• Define the research questions.
• Identify the data that will answer those questions.
• Select the research methods that will provide the
Understand The Real Problem
Fake Problem: “We
get too many calls to
the call center that
people should be
using the website to
organized poorly or
the site and the FAQ’s
are anything but.
How would we find
Define the Research Goals Correctly
Question: “Do Users
like the message
message boards the
best way of meeting
How would we find
Identify the Data that Will Answer
Do you need to understand what people actually
do or what their opinions are?
Select the Research Methods that will
Provide the Necessary Data.
Definitions: UX Methods
Usability-Lab Studies: participants are brought into a lab,
one-on-one with a researcher, and given a set of scenarios
that lead to tasks and usage of specific interest within a
product or service.
Ethnographic Field Studies: researchers meet with and
study participants in their natural environment, where they
would most likely encounter the product or service in
Participatory Design: participants are given design
elements or creative materials in order to construct their
ideal experience in a concrete way that expresses what
matters to them most and why.
Focus Groups: groups of 3-12 participants are lead
through a discussion about a set of topics, giving verbal
and written feedback through discussion and exercises.
A/B Testing (also known as “multivariate testing,” “live
testing,” or “bucket testing”): a method of scientifically
testing different designs on a site by randomly assigning
groups of users to interact with each of the different
designs and measuring the effect of these assignments on
True-Intent Studies: a method that asks random site
visitors what their goal or intention is upon entering the
site, measures their subsequent behavior, and asks
whether they were successful in achieving their goal upon
exiting the site.
Interviews: a researcher meets with participants one-on-
one to discuss in depth what the participant thinks about
the topic in question.
Eyetracking: an eyetracking device is configured to
precisely measure where participants look as they perform
tasks or interact naturally with websites, applications,
physical products, or environments.
Usability Benchmarking: tightly scripted usability studies
are performed with several participants, using precise and
predetermined measures of performance.
Unmoderated Remote Panel Studies: a panel of trained
participants who have video recording and data collection
software installed on their own personal devices uses a
website or product while thinking aloud, having their
experience recorded for immediate playback and analysis
by the researcher or company.
Clickstream Analysis: analyzing the record of screens or
pages that users clicks on and sees, as they use a site or
software product; it requires the site to be instrumented
properly or the application to have telemetry data
Intercept Surveys: a survey that is triggered during the
use of a site or application.
Email Surveys: a survey in which participants are
recruited from an email message.
You are an
designer looking to
make a breakthrough
medical app for the
In groups of 4 or 5, take 7
minutes to read the
information from Packet 1
What type of app do you have
Take an additional 7 minutes to
read the information from
What type of App do you have
in mind now? Did it change and
Using the right data in the right combination is crucial to
creating the experience you want your user to have.
A Website For
Senor Director of Technology, Columbia
Lighthouse for the Blind
Senior UX Designer, Brightfind
Color Contrast for Text
Text color must have sufficient contrast compared
to it’s background.
Text requires a contrast ratio of 4.5:1 for normal
text (<18px) and 3:1 for large text (>24px).
You can check the color contrast of text to its
background by entering the color codes in a
contrast checker such as the one at:
View graphics/websites as a color blind user:
• Adobe Photoshop or Illustrator [View > Proof Setup >
Color Blindness (Deuteranopia for red/green color
• Chrome plugin: Spectrum
WebAIM’s 2015 survey of screen reader users1 asked
participants if they utilized “skip navigation” links.
18.3% - Whenever they're available
19.5% - Often
30.2% - Sometimes
17.7% - Seldom
14.3% - Never
68% use skip nav functionality at least sometimes
Buttons on Mobile
82% of blind users reported using a screen reader
on their mobile device in 2014.1
Buttons on Mobile
“When some developers design their apps, they
don’t label all of their buttons and controls, so the
screen reader just says, ‘This is a button,’ but it
doesn’t say what the button actually does,” Mr.
Perez said. “That’s an area where we need a lot of
Testing for Accessibility
W3C Web Accessibility Evaluation Tools List:
NVDA is a free screen reader for Windows (they
request a donation)
Testing for Accessibility
AInspector Sidebar [for Firefox]
Testing for Accessibility
Navigate every web page using the keyboard only.
• Navigate with these keys:
o Tab (forwards)
o Shift + Tab (backwards)
o Space bar (sometimes used to select and unselect
o Arrow keys
o Enter key
Test that all links get visual focus (an outline) when they
are tabbed to, so that users who do not use a mouse
know they successfully landed on a link.
Testing for Accessibility
Check that all data tables have proper structure:
Check color contrast of text against background
Testing for Accessibility
Check for ARIA tags
View websites with styles disabled
• Web Developer Toolbar for Chrome and Firefox
This could hurt, huh?https://brailleworks.com/wp-content/uploads/2014/04/Braille-Fail-10.jpg
These failures are the result of a lack of empathy. Someone was told that they had to put a ramp by the door, but they didn’t stop and think about how that ramp would be used. I see similar mistakes in web development all the time. Developers know that they need to put certain things in their code to make it accessible or 508 compliant, but they don’t always have a clear picture of how these codes will be used.
This photo represents the biggest mistake in web accessibility. What went wrong here? How did this building end up with a wheelchair priority parking spot, with an icon of wheelchair on it, but without a RAMP for a wheelchair? You may wonder, how does this happen?
I don’t know where this building is, but I’m pretty sure I know what went wrong here. And the reason I know that, is because I’ve seen this same problem with websites for years. And that problem is saving accessibility for the QA process. You don’t want to do that. And here’s why. Look at this photo again. Imagine that you funded the construction of this building. And at the end you bring in your ADA tester, and she says you need to rip out those stairs and build a ramp. But that ramp is going to be too steep, so you need to level the ground first. So you need to pretty much dig up all the whole front here, and start over. And if you don’t have elevators in that building, you’re going to rip out some stairs and add them. That’s not the best way to spend your budget, right? It would have been better to plan for wheelchair access from the beginning, before you started construction.
You need to do the same thing with web design. And that’s because accessibility is much more than adding alt tags to images. If your website loads a pop up or a modal window, there’s a good chance it’s not accessible. It’s a poor usability practice, and it’s annoying to everyone, but it’s particularly problematic to your audience who is using a screen reader. They may not be able to easily close the popup, especially if the close icon is missing or not coded properly. I have a link to a video here showing a screen reader user struggling to close a pop up modal. https://www.youtube.com/watch?v=lrXDulundpE#action=share (story about this: https://motherboard.vice.com/read/you-think-popup-ads-are-bad-theyre-even-worse-for-the-blind). So now let’s say you’re using a popup to get people to sign up for your mailing list or your newsletter, but I tell you that it’s not accessible, now what? You need to rethink how you’re going to advertise your mailing list. A cake layer with a value proposition and a call to action will be much better. As will a prominent Call To Action in your utility navigation. Both of those options are much better for both accessibility and usability than a pop up modal window.
Accessibility is not just for people who are blind. Accessibility guidelines, for example the WCAG 2.0 guidelines, benefit people with physical challenges who cannot use a mouse. They help older members of audience who may struggle to read text that is too small or too light. Phyllis’ husband had a stroke. She is looking online for therapy options and for other information about how she can help her husband. She searches Google for occupational therapists and she finds a website listing them. But when she links to the website, she can’t read it. The text is too small and the font color is too light.
2015 football game between the Jets and the Bills
2015 football game between the Jets and the Bills. This is how color blind people see the game. It’s a little hard to tell the uniforms apart.
I’d like you to imagine that you have a 10 year old son named Jackson. Jackson is in 5th grade and he is learning how to conduct research on the web. One day his teacher gives his class a test. One of the questions on that test is to go a specific website and find this pie chart, and put the approximate values for fruit and vegetables on his exam. His classmates are all easily reading this, estimating that fruit comprises about 75-80% of the chart. They see the pie chart on the left. But Jackson is color blind. When Jackson looks at this, he sees the pie chart on the right which is pretty much just a green circle. Jackson comes home crying because he failed the test. How do you feel about that?
Now imaging that Jackson wasn’t your son, but that this pie chart was on your website. I don’t think any of us want this on our websites.
The most important improvement we made here is listing the text values along with the graph. Listing the data values separately from the graphic is required for accessibility and for 508 compliance, and it is also improves the usability for everyone. It’s hard to gauge that the pie chart is split into 20% and 80% slices. Can you really tell from looking if that smaller slice is 20% or if it is 75%? Listing the value makes it clear to everyone. It’s helpful to add a border (white or black) to separate two adjacent colors in graphs. We alternated the saturation of the two colors help differentiate the contrast of both slices.
Parece que tem um bloqueador de anúncios ativo. Ao listar o SlideShare no seu bloqueador de anúncios, está a apoiar a nossa comunidade de criadores de conteúdo.
Atualizámos a nossa política de privacidade.
Atualizámos a nossa política de privacidade de modo a estarmos em conformidade com os regulamentos de privacidade em constante mutação a nível mundial e para lhe fornecer uma visão sobre as formas limitadas de utilização dos seus dados.
Pode ler os detalhes abaixo. Ao aceitar, está a concordar com a política de privacidade atualizada.