O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

Usability Testing and QA 12 11-15

Carregando em…3

Confira estes a seguir

1 de 78 Anúncio

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a Usability Testing and QA 12 11-15 (20)


Mais recentes (20)

Usability Testing and QA 12 11-15

  1. 1. Usability Testing and QA A practical approach towards creating better user interfaces
  2. 2. Who am I? Shilpa Thanawala @skthana
  3. 3. And you? Improving usability is about listening, shifting your perspective, empathy. Interview your neighbor, then introduce her/him. Name Role at your company or agency Background (eg. designer? developer? etc.) Experience with usability testing How you think you’ll use what you learn today Why this class?
  4. 4. Housekeeping Initial and sign the roster Evaluations at the end of the day Cell phones on silent, calls outside Breaks & lunch
  5. 5. Usability Testing Some thoughts to start us off...
  6. 6. Designers know too much about their product to be objective judges; the features they have come to love and prefer may not be understood or preferred by future customers. Don Norman, nngroup.com Image credit: templatemonster.com
  7. 7. The test of the machine is the satisfaction it gives you. There isn't any other test. If the machine produces tranquility it's right. If it disturbs you it's wrong until either the machine or your mind is changed. Robert M. Pirsig, Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values Image credit: Optimum7.com
  8. 8. Supposing is good, but finding out is better. Mark Twain Image credit: Mediamatic
  9. 9. Usability Testing Some historical highlights...
  10. 10. Early 20th Century Improvements in industrial efficiency In WW I, reduced work motions for soldiers to assemble & take apart weapons in the dark. http://www.measuringusability.com/blog/usability-history.php
  11. 11. 1947 John Karlin - how short phone cords should be
  12. 12. “It is not so much that Mr. Karlin trained midcentury Americans how to use the telephone. It is, rather, that by studying the psychological capabilities and limitations of ordinary people, he trained the telephone...” Excerpt from New York Times article
  13. 13. 1980’s New methodologies developed and published Usability becomes a profession Defined as a function of efficiency, effectiveness, and satisfaction first publishedPsychology of Everyday Things 1990’s Usability matures Diminishing returns from testing more than 3-5 participants Books by Jakob Nielsen
  14. 14. 2000 - today Steve Krug’s Don’t Make Me Think Remote testing Statistical analysis of usability data Mobile devices, touch and gestures http://www.measuringusability.com/blog/usability-history.php
  15. 15. Why?
  16. 16. Why spend valuable time and money doing usability tests? It's the best way to learn how users actually interact with your website. Watching > interviewing You don't really know your users' needs. Users are not all the same. And they're not your Mom. All websites have problems, and the serious ones are easy to find.
  17. 17. Why spend valuable time and money doing usability tests? You'll learn things you didn't know that you didn't know. Watching users gets everyone on the same page. Watching users will make you a better designer / developer / product manager / content writer / etc. Usability testing doesn't have to be expensive or time- consuming.
  18. 18. What Testing Can't Tell You
  19. 19. Limitations of Usability Testing Won't fix your site's problems (it's not a user training session) Results are dependent on the questions you ask (tasks) Can't model social interactions very well Isn't always representative of real situations For more on issues testing the social web see Dana Chisnell's talk
  20. 20. When?
  21. 21. When should usability testing be done? Before you design or build anything Before you implement a change Throughout the project After implementing a fix Throughout the life of the website As soon as possible & repeat often!
  22. 22. Demo: A short usability test
  23. 23. Qualitative vs. Quantitative
  24. 24. Qualitative vs. Quantitative Quantitative Usability Studies Designed and conducted to produce data for statistical analysis Each measurement represents a count or an amount Distances, weights, quantities, etc. Qualitative Usability Studies Focused on verbal descriptions of users' experiences Each measurement is a description or category Words, sentences, feelings, yes/no Categories are not quantitative, even if they are assigned numbers userfocus.co.uk
  25. 25. “...qualitative methods are much better suited for answering questions about why or how to fix a problem, whereas quantitative methods do a much better job answering ‘how many’ and ‘how much’ types of questions.” More on Quantitative methods at "When to Use Which User Experience Research Methods", nngroup.com measuringusability.com
  26. 26. Usability Metrics
  27. 27. Effectiveness, Efficiency, and Satisfaction Effectiveness Accuracy, correctness. Are users able to achieve their goals? Efficiency Ease of use, speed. How much effort (time) does it take to complete a task? Satisfaction Users' perception. What does the user think about their experience? More at ,usability.gov wikipedia.org/wiki/Usability
  28. 28. Some Metrics for Common User Tasks From Measuring the User Experience – Tullis & Albert
  29. 29. Performance-based Metrics Task Success Commonly used, pass / fail, or degrees of success, should have clear end-state. Task Time Particularly important for repetitive tasks Errors Incorrect actions leading to significant time loss, additional costs, or task failure Efficiency Effort and time needed to complete the task Learnability Effort and time needed for the user to learn how to use the interface
  30. 30. Issue Metrics Usability findings based on users' behavior Consistent / repeatable, or idiosyncratic? Remain open-minded to the unexpected
  31. 31. Other Metrics Self-Reporting: Users' perceptions, feelings Behavioral / Psychological: Eye-tracking, stress, unprompted verbal expressions Combinations / Comparisons: Interpretations based on more than one metric Card-Sorting: How to organize information in a way that makes sense to users. A/B Testing: Comparison of two alternate designs Accessibility: Usability for those with disabilities ( , ) CCVA of 2010 WCAG
  32. 32. Bias Every study contains some level of bias. Participant background, knowledge, comfort level Task definitions Methodology (testing process, session length, how much talking) Artifacts of the prototype or product Environment (lighting, noise, cameras, distractions) The Facilitator (experience, skill, style) Expectations Measuring the User Experience – Tullis & Albert
  33. 33. Number of Test Participants
  34. 34. What's the right number of users to recruit for a study? 3-5? 4-6? 10? 20? 100? Those who favor a small number feel 80% of the most important usability problems are detected by the first few users. Cases in which more users may be required: You need to be sure you've captured as many problems as possible You have more than one user group (persona) You have a large number of screens, complex design, wide variety of tasks You'd like to account for evaluator bias
  35. 35. Why we'll focus on qualitative testing today Anyone can do it No knowledge of statistics or data analysis needed; based on verbal feedback, observation, and empathy Cheap, fast, and easy Great for small budgets, busy teams, minimal resources... and frequent repeats It gets the job done More than sufficient to identify a website's most serious usability problems
  36. 36. Break
  37. 37. DIY Usability Tests Based largely on Rocket Surgery Made Easy – Steve Krug
  38. 38. Establish your Goals What do you want to learn about your site? Can users find products easily? Is it clear to users what the site is about? Is the information organized intuitively? Is this new feature distracting? Keep an open mind to allow for unexpected results.
  39. 39. Define your Tasks Tasks → Activities Determine which activities best target what you want to learn. Create one or more scenarios based on essential, real-world user goals. Then, describe the activities in the context of your scenario. Activities should be realistic Activities should be actionable Phrase them so as not to give away the answer Print each activity on its own piece of paper. (Avoid labels or numbers.) "Turn User Goals into Task Scenarios for Usability Testing" — nngroup.com
  40. 40. Recruiting Test Participants What kinds of participants? Where to find them? How many? How to compensate them?
  41. 41. What kinds of participants? How representative of actual users? Most serious UI problems will be discovered by non- representative users. Testing with non-representative users is better than obsessing over finding representative users. Key requirements: not part of the team, fluent in the UI's language
  42. 42. Where to find participants? Where do your users hang out? Advertise on the site, message boards, social networks Craigslist
  43. 43. How many users? For most of us doing DIY testing, 3-5 participants is fine. No-shows or last-minute cancellations are common. Have 1-2 backup participants lined up. Phone-screen, scheduling, day-before reminders, directions, parking, NDA's... use a checklist
  44. 44. Compensation Shows appreciation for participant's time and effort Encourages participants to be engaged and enthusiastic. Gift cards, free product or service.
  45. 45. Equipment & Setup Hint: you don't need a lab Bare minimum (if you're the only one): a computer in a quiet room, a pen and notepad, and 2 chairs For a team, add a remote observation room, a good mic, and screen-sharing (Skype or GHangout). And provide irresistable snacks. Can use screen-capture software (but most of the time it won't be watched) Can record the participant (but it's not really needed)
  46. 46. Make it a Team Effort Try to get everyone to attend. Assign a trustworthy team member to manage the observation room
  47. 47. The Session Script Start with a template & edit to suit your needs Welcome / introduction Logistics, release forms Necessary background information Test: general questions Test: scenario activities Wrapup, follow-up questions Conclude and provide honorarium
  48. 48. Some Test Session Script Templates Free templates at: , ,infodesign.com Steve Krug's site Book site guide by Rubin, Chisnell & Spool
  49. 49. Debrief Only for those who attended at least one session Schedule it on the same day (next day latest), 60-90 min. Provide food! Get all observers to state (or stick) UX problems they observed Get general agreement on which are most serious & will be tackled first (2-3 maximum) Remind everyone when the next round of testing will take place
  50. 50. Run your own usability test 1. Select a website to test 2. Determine goals: What's the question you're trying to answer in this round of testing? 3. Create a scenario & 3 activities that will give you information pertaining to your goal
  51. 51. Lunch 1 hour
  52. 52. Run your own usability test 1. Prepare your script (links to resources ) 2. Rehearse and refine 3. Print scenario(s) and activities on 1 sheet (well-spaced so you can cut them apart) j.mp/ux-dec-11
  53. 53. Run your own usability test 1. Test two participants (~20 min each) 2. Take notes
  54. 54. Run your own usability test 1. Debrief within your original group 2. Identify 2-3 usability issues
  55. 55. Tools Testing session checklists , , Usability testing software , , Online / Remote services , , , Usability.gov Krug's checklist checklist on book site by Rubin, Chisnell & Spool Silverback (Mac) Morae (Win) Camtasia Usertesting.com Loop11 usabilityhub.com many more
  56. 56. So how do we fix it?
  57. 57. Fixing usability issues The bad news: most issues never get fixed Make the smallest change possible to "stop the bleeding" Large changes might introduce new issues It's not fixed until you verify by ... testing again!
  58. 58. Demo: Prototyping a UI change using Photoshop + InVision One method (of many) to prototype a change: Create some static mockups using screenshots and visual editing software Model interactions using the InVision app
  59. 59. A/B Testing
  60. 60. A/B Testing A simple experiment testing two cases. Before vs. after, one fix vs. another Qualitative or quantitative
  61. 61. A/B Testing Possibilities Layout and design Relative sizes of UI elements UI controls Form elements (how many, field labels, helper-text) Images Copy length or content Calls to action ...many more
  62. 62. A/B Testing Activity Prototyping your proposed fixes
  63. 63. A Usability Test using Paper Prototypes Paper prototype of a kids’ website Credit: BlueDuckLabs / YouTube.com View online
  64. 64. A/B Testing Activity 1. Select one usability issue from your previous activity 2. Brainstorm at least two ways to fix it 3. Print the page and use either Balsamiq or paper prototyping to mock up your two solutions. 4. Test each case on a separate participant in the other group 5. Debrief in your group. Which worked better, A or B? More detail at: SM's Ultimate Guide to A/B Testing
  65. 65. Testing Accessibility
  66. 66. Why is accessibility important? “The Web is an increasingly important resource in many aspects of life: education, employment, government, commerce, health care, recreation, and more. It is essential that the Web be accessible in order to provide equal access and equal opportunity to people with disabilities. ” From The W3C's Web Accessibility Initiative
  67. 67. Is my site accessible? Unplug your mouse / turn off your trackpad Why - blind, low-vision, compromised motor-control, no hands How to test - start at url, use tab / shift-tab and arrows to see whether you can access all controls, links, and fields on the page, in the right order. Popups and dialogs should be usable. How to fix - proper HTML heirarchy, tabindex attribute, javascript plugins
  68. 68. Is my site accessible? Check color contrast Why - low-vision; 1 out of 12 have color deficiency How to test - online tools ( , ), browser plugins ( ) How to fix - shift color palette to improve contrast, increase font sizes for low contrast areas, avoid colorful background images Check my Colors Contrast Ratio Grayscale Tool
  69. 69. Is my site accessible? Turn off images Why - images useless to low-vision or blind users, low bandwidth How to test - block images in browser, WebAIM tool ( ) How to fix - proper alt attributes, don't use images when text can be used WAVE
  70. 70. Accessibility Resources and Tools WebAIM W3C's Web Accessibility Initiative Quick accessibility tests & fixes SitePoint's Easy accessibility checks
  71. 71. Mobile / Touch
  72. 72. Mobile Usability Context / environment (noise, lighting, glare) Many more contexts beyond desktop Bandwidth Mobile devices force us to think about bandwidth again Touch target size Average fingertip size = 10mm Adjustments in spacing, button size Touch is also being used on large screens Driving / attention What usability issues contribute to driver distraction?
  73. 73. Mobile Usability Testing Resources NNGroup recommendations Article on UXMagazine
  74. 74. Break 10 minutes
  75. 75. Accessibility Testing Activity 1. Click the URL at the top of the browser window 2. Without using the mouse again, try using only the keyboard to navigate 3. Can you access all the links, menus, and controls? 4. Can you determine where you are, or is it easy to lose your place?
  76. 76. Wrap-up Usability testing is an essential tool for improving your website or product Anyone can do a simple usability study Test as early as possible, as frequently as possible Tailor the type of testing you conduct (qualitative / quantitative, metrics, number of participants) to give you the information you're after Use the available tools and resources to help you run your sessions
  77. 77. Wrap-up Get your whole team involved Small adjustments are preferable (and faster) than a full redesign Remember to test your fixes, too Stay aware of non-standard users (accessibility) and the changing technological landscape (new platforms, new uses)
  78. 78. Resources Websites / Blogs: , , , Quantitative Testing: , , nngroup.com usability.gov sensible.com (Krug) UIE measuringusability.com Measuring the User Experience – Tullis & Albert usability.gov Thank you! Shilpa Thanawala | @skthana