Usability testing is a fantastic way to get customer feedback and easy enough to execute that anyone can do it. At this session we'll walk through a very quick introduction to testing, how to test (even with a budget of $0), and run through some case studies of successful tests. Attend the workshop if you’re looking to gather meaningful feedback, develop front-end facing applications, work alongside a UX team, or you’re just interested in learning more.
5. Who am I?
• Principal UX Consultant for SafeNet
• Co-founder of Brew City UX
• Adjunct UX Professor at MSOE
• Occasional blogger for That Conference
• @ashleydzick on Twitter
6. Who are you?
• Developer, UX, BA, PM, etc.
• Interested in gathering user feedback
• Want to understand usability testing
• May not have a UX team
• Definitely does not have a budget
7. What is UX?
• Basic Usability
• Interface
• Workflows
• Interactions, Gestures
• Errors, Messaging
• Colors and Overall Aesthetics
• Responsiveness
• Performance and Speed
• Software Bugs and Reliability
• Content, Voice and Tone
• Brand Personality
• Images, Video and Media
• Customer Service
• Everything that impacts the user
9. “Up to 15% of IT projects are abandoned and at
least 50% of a programmers’ time during the
project is spent doing rework that is avoidable.”
Dr. Susan Weinschenk for HFI
10. “60% of customers who’ve had a poor customer
service experience are unlikely to return even if
a trusted friend tells them service has improved.”
Kayako
11. “Over 1 trillion USD is spent on IT projects
annually, but 15% of those projects will be pulled
before completion… 25% of those are related to
UX mismanagement.”
Justinmind
12. Okay… but why YOU?
• Get feedback that isn’t your own
• Develop meaningful interactions
• Iterate quickly through different solutions
13. What can you usability test?
• Basic Usability
• Interface
• Workflows
• Interactions, Gestures
• Errors, Messaging
• Colors and Overall Aesthetics
• Responsiveness
• Performance and Speed
• Software Bugs and Reliability
• Content, Voice and Tone
• Brand Personality
• Images, Video and Media
• Customer Service
14. Cost v. benefit of testing
Outside
Perspective
Verify
Solution
Idea
Iteration
Complex
Refinements
Benefit
Cost,Formality,Effort
15. What about analytics?
• Tells you what you need to look at, not why
• e.g. “Are they spending a long time on this page because it is
confusing, or because it is important?”
18. • What are you trying to get feedback on?
• Test any assumptions, opinions or ideas
• Pick your method & determine scope
Theorize hypothesis
19. • People around you
• Potential customers
• Beta users
• Tag along on marketing trips
• Recruitment firm
• Anyone is better than no one
Recruit participants
20. • Right degree of ‘realness’
• Account for every step in test workflow
• In person or remote
• Recording equipment
Tip: you don’t have to record, but it helps you with your notes later on
Establish test
21. • Brief your users
• Run them through your scenarios
• Ask clarifying questions and capture results
• Pay attention to what they do more than what they say
• Thank them for their time
Tip: don’t let users get distracted by fake data
Execute test
22. • “We aren’t testing you, we’re testing [product/workflow/app]"
• People innately want to do well
• Tell them it is okay to fail
• Explain why you are testing
• Explain how long it is going to take
Briefing your users
23. • Review your notes & recordings
• Discuss ASAP with anyone else who observed
• Where did users get snagged?
• What assumptions did they make?
• Do you see any themes or trends?
• Who should see the results?
Socialize findings
25. Cost v. benefit of testing
Outside
Perspective
Verify
Solution
Idea
Iteration
Complex
Refinements
Benefit
Cost,Formality,Effort
26. Right-size your test
$0 and 0
time
$0 and
other’s
time
$0 and my
own time
Large
investment
Outside
Perspective
Verify
Solution
Idea
Iteration
Complex
Refinements
Benefit
Cost,Formality,Effor
27. Right-size your test
$0 and 0
time
$0 and
other’s time
$0 and my
own time
Large
investment
Outside
Perspective
Verify
Solution
Idea
Iteration
Complex
Refinements
Benefit
Cost,Formali
Show and
ask
Guerilla
testing
Bells and
whistles
Regular ol’
testing
28. Right-size your test
Show and
ask
Guerilla
testing
Bells and
whistles
Regular Ol’
testing
Benefit
Cost,Formality,Effort
29. • Barebones beginning testing with $0 and little time to spare
• Initial idea generation
• Very informal feedback
• Test with anyone besides yourself and your team
Show and ask
34. Show and ask
T: Opinions
R: Test with any (alive) human
E: Just need your device
E: Show device, ad-hoc
S: Report is in your head
35. Tools of the trade
Prototyping: Basic HTML/CSS, Images, Sketches
Notes: In your head, pen and paper, record it with your phone
Nothing else really required!
36. Guerilla testing
• $0, but you’ve got some time to invest
• Actual users (if possible)
• Have a hypothesis or scenarios
• Let the user drive
• Might invite others to watch testing
Tip: gorillas make terrible users to test
39. Guerilla testing
T: Have a workflow or scenario but still informal
R: Test with anyone in your user base or similar
E: Just your prototype
E: Give them the device and let them play
S: Informal discussion
40. Tools of the trade
Prototyping: Paper, InVision, Basic HTML/CSS, Images
Notes: Pen and paper, sticky notes, google docs (collaborative)
Recording: Phone or webcam
41. Regular ol’ testing
• $0 to low cost
• Most common type of formalized user testing
• Actual users or potential customers
• Testing plan with tasks or scenarios
• 5-7 users for meaningful results
47. Regular ol’ testing
T: Specific questions and scenarios
R: Meaningful segment of your users, invite audience
E: Prototype or working software, record session
E: Explain testing process, ask questions and clarification
S: Highlights, important pieces, quotes, video clips
48. Tools of the trade
Prototyping: Working Software, Axure
Notes: Pen and paper, sticky notes, google docs (collaborative)
Recording: Phone, Webcam, Conferencing Software
Documents: Testing plan, contact info for users, schedule
Legal: Consent form, Non disclosures, Recording agreement
49. • $$$
• Complex, ongoing user research strategy
• Hire a service or have in-house dedicated researchers
• Incentives for user participation
• Consent forms, NDAs, other legal requirements
• Avoid lawsuit, acquisition, new product or market
Bells and whistles
53. Bells and whistles
T: “How can I completely optimize the customer experience?”
R: Large group of targeted current and potential users
E: Custom mobile sled, Cameras, Testing Suite
E: Usually use an external firm
S: Formal reports with itemized findings
54. Tools of the trade
• Data: Mockaroo, UINames, GenerateData, PlaceIMG
• Prototypes: HTML, Axure, InVision, MockFlow, Working Software
• Testing Suite: User Zoom, usertesting.com, Morae or Silverback + User Recruitment Firm
• Note taking: Pen & Paper, Sticky notes, Google Docs, Evernote, Onenote
• Organizing feedback: Trello, Realtimeboard, Spreadsheet
• Video conferencing: GoTo Meeting, Join.Me, WebEx, Google Hangouts
• Documents: Testing plan, contact info for users, schedule, results report
• Legal: Consent form, Non disclosures, Recording agreement, Sign off from legal team
55. Sizing your test
Just want some feedback = Show and ask
Introducing testing to your company = Show and ask, Guerrilla
Introducing someone else to testing = Guerrilla
Trying to get a testing budget = Guerrilla, Regular ol’
Statistically significant feedback = Regular ol’, Bells and whistles
A lot of money / avoiding a lawsuit = Bells and whistles
56. Most companies aim for here
Show and
Ask
Regular
Ol’ Testing
Guerilla
Testing
Bells and
Whistles
Benefit
Cost,Formality,Effort
57. Now what?
• Leverage the data
• Inform others
• Start a regular cadence
• Get a budget
60. • Non disclosures
• Camera or recording agreements (if you are going to share recording)
• Consent form
Tip: usability.gov has some great (but wordy) templates
Legal Forms
61. Learn more
• Why you only need to test with 5 users
• Usability testing consent form
• Intro to user testing on a budget
• Usability test, even when you know the answer
