2. Before we begin…
1. What do you already know about
usability testing?
2. What do you want to learn today?
3.
4.
5.
6. Limitations of Opinion
“The accuracy of subjective reports is so poor
as to suggest that any introspective access
that may exist is not sufficient to produce
generally correct or reliable reports.”
- R. E. Nisbett and T. D. Wilson, On telling more than
we can know: Verbal reports on mental processes,
Psychological Review, Vol 84. (1977)
7. To improve a product…
1. Reliable data is crucial
8.
9.
10.
11. To improve a product…
1. Reliable data is crucial
2. Need to find out user expectations
12. Usability =
The ease with which people can
employ a particular tool or other
human-made object in order to
achieve a particular goal - Wikipedia
14. Your turn: What sites have
these?
Effective
Efficient
Engaging
Error Tolerant
Easy To Learn
15. Usability Testing =
Setting a series of tasks for people to
complete and noting any problems
they encounter
– Andy Budd
16. What‟s involved in a usability
test
• Participants performing meaningful tasks
• Real users – not your BFF
• Observers (you or design team)
• Recording data for analysis
17. You can test ANYTHING
• Hand-drawn sketch
• Wireframe
• Screenshot / JPG
• HTML prototype
• Live site
• Competitor‟s site!
18.
19. Why care about usability?
• Simple errors can lead to big problems
• If the customer isn‟t happy, nobody‟s happy.
• If visitors can‟t easily access your info, why have
a site at all?
• Casual visitors have a different experience than
those who work with a website every day.
24. What‟s the ROI?
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails
25. What‟s the ROI?
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails
• Higher task completion rate
26. What‟s the ROI?
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails
• Higher task completion rate
• Increased productivity
27. What‟s the ROI?
• Reduced developer time
• Reduced training time
• Reduced help desk calls / emails
• Higher task completion rate
• Increased productivity
• PR boost
28. Your turn: Any usability
problems?
• Break into groups
• Three usability problems you‟ve seen
• 5 min.
29. Common usability problems
• Too many words
• Excessive “insider speak” or jargon
• Critical tasks hard to find
• Poor navigation
• Irrelevant search Results
37. Formal Informal
Recruiting, testing done for DIY
you
Trained Facilitators Facilitators-in-training (i.e.
You)
Smaller chance of bias Larger chance of bias
Lengthy to set up + Fast setup, use
implement immediately
Gorgeous Reports Word Templates
Expensive ($5k and up) Cheap / free
45. Top Tasks – Why they came
here
• Order a pizza
• Find your contact information
• Register for a class
• Leave a comment
• Get a rate quote
• Learn about your services
46.
47.
48. Active Listening / Think-Aloud
Protocol
• Tester does 90% of talking
• You = Therapist / Travel Agent
• Open Ended Questions (if any)
• Non-leading responses – “Thanks for the
feedback!”
• Take notes but give attention
For my usabiltiy presentation to I Want to picture in your head hanging out with a hollywood celebrity. Famous, talented AND instantly recoginzable- two Screen Actors Guild Awards, a Golden Globe, and two nominations for an Emmy Award. Picture that person in your head, OK?
We love our baby – it’s beautiful
But what if it’s an ugly baby? No one ever tells you if you’ve got an ugly baby. Sam – ugliest dog in world
Aha! We’ll get feedback. We’ll do focus groups, questionarries, surveys and we’ll get everything we need to know.
Problem – subject information has limitations. Eye witnesses so unreliable as to often not hold up in court. Easily influence –”do you remember brokejn glass wen it hit you – biased onb the misfornaiton effect.
GSA advantage – billions of dollars – tractors to notebooks to consulting services - “We are not Amazon”
The leading sites create the user experience that the world expects. You either meet these expectations or people get angry.
Meeting peoples needs and expectations is what user experience and usability are all about. Can be the largest factor in success – between a Zune and an ipod
Market reseach is what people say, usabiltiy is what they do. Gain feedback from customers and improve system.
Market reseach is what people say, usabiltiy is what they do. Gain feedback from customers and improve system.
Market reseach is what people say, usabiltiy is what they do. Gain feedback from customers and improve system.
Study them! Don’t rip off indiscrimnately, but look for good ideas
Market research is what people say, usabilityis what they do. Gain feedback from customers and improve system.
Behaviour is observed — what users do (rather than what users say they do).Participants perform real tasks — tasks that are meaningful to them and to the business.Involves real users — not just someone at the next desk.Observations and data are recorded and the data are analysed.
Getting feedback from customers as they use your site
Training – is failure. Everyone hates training. Trianing means your product is hard to use. Ever gotten training for your ATM? Or to turn on a car? You see it once, you can do it.
Ok, maybe making a product usable isa good idea, but provide it.
Study them! Don’t rip off indiscrimnately, but look for good ideas
Study them! Don’t rip off indiscrimnately, but look for good ideas
Here’s one story:Tested a site recently that get in the press it was so bad. When did they do usability testing?
Testing – we do testing. We do Quality assurance testing. Here if not earlier
My dad send this fax in the 1970s
People skip because they think they know what their users are thinking and expect. But they don’t. you don’t. I don’t.
guerilla - one who fights with few resoruces, quick and dirty, guided by principleguerilla UCD - not easy, not perfect, not for everyonegueriliaauCD - is CHEAP, fast, effective
Hallway test – more users, informal, public space. Common room, coffee shop
you don’t need a huge amount of people to identify a problem (use my slide about 70%)
95% of usability tests are discount – small sample sizes of 10 or fewer, maybe around 5.
Qualitiative testing with small sample size – 3-5 per day is max
From the oatmeal.com
From the oatmeal.com
makes other people feel important and smart, open ended questions. Mostly listening but managing conversation – restate what they are saying, non-leading questions. “Thank you for that feedback” rather than “Correct! You found it”
Test with usability.gov – purpose, audience, what services they provide, contact them, any cost, free tools or materials,
Always end each meeting with a debrief session, discuss finding of the day while still fresh in everyone’s minds
Usability is now a team sport - observers are a part of the process, stakeholders are a part – more buy in
Sites to test:
Small changes can have big impact. Army – 30% decrease in calls. To get this impact, we need to make you successful, and to make you successful we give people a deadline of 30 days to make the changes we reocmmend