4. Refer a friend! Know someone
who would be perfect for one of our
open positions? $2500 is yours if
your referral is hired
We’re Hiring!
lisa.rex@acquia.com dharmesh.mistry@acquia.com
17. GENERAL CONSTRUCTION
1. Goals
2. Prepare
- Recruit (Moderated)
- Conduct (Moderated)
- Set up (Unmoderated)
3. Watch the videos & analyze
4. Report findings
18. PREP STEP 1: GOALS
Talk to stakeholders:
“What are the business goals of the study?”
“What do we need to learn?”
“What are our primary tasks & personas?”
“When do we need it?”
“What’s our budget for usability testing and fixing
problems?”
19. PREP STEP 2: WHAT TO TEST?
Information
architecture
Prototype
Staging site
Production site
You can test at any point during the project
lifecycle
20. PREP STEP 3: PARTICIPANTS
Who to test?
New users vs. current users
Primary roles & personas
How many to test?
5 participants per study is ideal for uncovering
the major usability problems while remaining
quick and cost effective.
15+ participants for a card sort / IA study
21. PREP STEP 4: CHOOSING THE METHOD
Variety of webapps
Usertesting.com, Verifyapp, Treejack,
etc
22. PREP STEP 5: STUDY OUTLINE
Starting URL
Scenario
Tasks (include login details)
Post-session questions
Choose your participants
http://bit.ly/views-study
23. PREP STEP 5a: WRITING TASKS
Do not use words that are in the UI
Phrase things in the way people think
Write in plain language
Put tasks in order; dependencies between the
tasks?
Include assists
Ask them to rate their experience
24. PREP STEP 5: TEST SITE ENVIRONMENT
How will they access the site?
Content and user account setup is
needed?
Do you need to set up separate sites
for each? Or re-set between sessions?
Tip: Use can use Drush to disable and uninstall modules!
Or, if you have a dev, stage and prod environment,
overwrite the database on the testing environment
25. Starting URL: drupal.org
Scenario: You have built sites in other
CMS, have heard of Drupal, and would like
to know more. You work for a non-profit
that is looking to replace it’s site with an
open source solution.
26. Task 1: Explore for a couple minutes and tell me
what you have learned about Drupal.
Task 2: You want to find Drupal experts for hire.
Show me how you would do this.
Task 3: If you didn’t find it already, Marketplace
lists Drupal companies. Now, you want to learn
more about the Drupal community. What would
you do?
Task 4: Let’s say you want to post a support
question in the forums. Tell me what steps you
would take.
27. PREP STEP 8: OBSERVATION FORM
Priority/Severity
Issue summary
Link to screenshot
Task name
Participants who had the issue (frequency)
29. STEP 5: WATCH THE VIDEOS
What to note?
Things that went well
Things that did not go well
Capture quotes and annotate screenshots or get
video clips
30. STEP 5: ANALYSIS, cont.
Categorize things that did not go well into bugs
and usability issues
Severity: Low/Medium/ High (or
1-5)
Frequency: Number of occurrences
Scope: Local/ Global
31. STEP 6: THE EXECUTIVE SUMMARY
Overall: There were no Critical problems
uncovered. Overall, the response was positive
and participants could find their way towards
achieving the tasks quickly.
High severity findings
• Bulleted list
Medium severity findings
• Bulleted list
The detailed findings can be found at [URL].
32. Tools for testing live
sites
- Usertesting.com
- Userlytics.com
- Loop11
Tools for testing
screenshots
- Five Second Test
- Invision app
- Verifyapp
Other
- Treejack (IA studies)
- Webform.com (Diary studies)
Acquia product usability , I’ve been in the community for just over three years, involved with Drupal.org projects, as well as a bit of work on Drupal 7 help text, and more recently we did Views usability study and a Drupal 8 information architecture usability study\n
\n
http://www.acquia.com/careers\n
LR\nHow many call themselves Developer/ Designer / Business Person, None of the above?)\nHow many of you have conducted usability testing before? \nHow many of you have asked for budget for usability studies?\n\n
LR - odon’t read this. we want to ensure you can run a usability study by the end of this session. overview of tools and process\n
\n
The way we design is not always the way things are used. Find out the delta betwenen them.\nThings we could uncover: \n Visual problems (like contrast)\n Error messaging and UI text\n Content\n Navigation and findability problems\n Workflow problems\n\n
\n
\n
DM\n
\n
\n
\n
“Usability testing is the second most used evaluation method and the method that has the most impact on making products better” \nAdvantages of unmoderated over moderated. \nUnmoderated is faster, asynchronous, less steps to cover but doesn’t offer probe and not suitable for studies that require a lot of in-time probing. \n\n
DM start\nPreparation is the longest phase of the process \n
\n
business goals translate to UX goals \n
\n
Usability specialists believe it is not cost effective to run more because it only uncovers more minor \nproblems\n\n\n\n“Virtually all of the problems judged high in severity were discovered after running 5 subjects, whereas only 55% of the problems judged low in severity were found.”\n
