Kirk Doggett and Kate Lawrence presented on using the usertesting.com tool for unmoderated user testing. They demonstrated a test of Vistaprint's business card design page. They discussed that usertesting.com allows recruiting specific users, testing websites and prototypes, and getting results quickly. However, tasks must be very clear as tests are unmoderated, and it may not be suitable for complex or time-consuming tasks. They highlighted when usertesting.com is appropriate to use, such as for targeted tasks, and its benefits like testing from anywhere.

1. Kirk Doggett
Manager, Global UX Research, Vistaprint
@GR8UX
#FLASHTEST: User Research Live
UXPA Boston, 2014
Kate Lawrence
Director, User Research, EBSCO Information Services
@bykatelawrence
Kirk

2. What you will learn today about the
usertesting.com tool:
We’ll launch a test, and while
it’s running, tell you how
Vistaprint and EBSCO
structure their tests and
report on the findings.
One size does not fit all; we’ll
tell you when it is appropriate
to use a tool like
usertesting.com and when
you are better off with
another method.
Why un-moderated testing
tools should be a part of your
research toolkit, and why
Usertesting.com works for
Vistaprint and EBSCO.
Why we like it When to use it How to use it

3. 3, 2, 1.…. Launch!

4. • Single largest player in the internet printing market,
helping small businesses around the world create
professional marketing products at an affordable price
• 25 localized websites
• 17 million customers
• 3 state of the art manufacturing plants
• 13 offices worldwide and growing
Kirk

5. User Research at Vistaprint
How:
• In-person testing
• Remote testing
• UT.com
• Phone Surveys
• Field Research
• Customer Council
• M-Turk
What:
• Our site
• Competitors
• Prototypes
Kirk

6. • Premier provider of research databases, eJournals
and eBooks worldwide
• Powerful discovery tools & management resources for
libraries and other institutions
• Point-of-care medical resources and corporate learning
tools
• We’re hiring! www.ebscohost.com/careers
Ipswich location, electric car charging stations & river views
Kate

7. User Research at EBSCO
Primary Research:
Asking the users ourselves
Data/
Usage Analysis:
What do the metrics show?
Review of
Secondary Research:
What did others learn?
Kate

8. Broad Selection of Tools
Kirk and
Kate
+

9. What we like about Usertesting.com
 It’s fast – results are typically back within 1 hour.
 You can recruit specific users – gender, age, country,
income, web expertise, other (must be a small business owner,
must be a college student, etc)
 You can provide a URL where you want users to start – can be
a site, dropbox, sharefile, etc.
 Test anything – powerpoints, prototypes, sites (yours, competitors)
 Give users 15 minutes worth of work
 Have 4 free-text questions after the tasks are complete.
Kate

10. What We’re Testing Today

12. Test Details
 Website URL: http://www.vistaprint.com/business-cards.aspx
 Scenario: Imagine you need to get business cards to represent yourself and your profession.
 Tasks:
1. Familiarize yourself with this page, and mention your first impressions.
2. Click the “Browse our designs” section to continue.
3. Look carefully to find a business card that appeals to you and represents you/your business; Find
one you would *really* want to buy.
4. After finding the design you like best, stop at the point where you begin to customize the card.
Comment on the process of finding a design.
 Questions for Participants:
1. What options or tools did you notice to help find your design? Was there anything missing?
2. What was the best part about finding a design?”
3. What frustrated you most about the process of finding a design?
4. What is the one most important improvement this site could make to make it easier to find a
design?
 Participant Demographics
 Gender – 50% M; 50% F
 30 to 60 years of age
 Country: US
 Income range from $40K+
 Web expertise: average web user
 One other such as “must be a small business owner”

13. Power Tip: The Pilot Test
• Use a single session to test the clarity of your instructions
• How well does the panelist respond?
• Are you learning what you set out to learn?
• If the pilot works, then run the rest of the sample
• If not, modify the test, and pilot again

14. Vistaprint: One-Week Test Plan
Monday
• Kickoff
• Goals
• Expectations
• Budget
Tuesday
• Starting URL
• Scenario
• Tasks
• Questions
• Demographics
Wednesday
• Refine plan
• Sign-off
• Staging site
• Prototype, etc.
Thursday
• Pilot
• Launch
• Analysis
• Begin report
Friday
• Report results
UX Research
Business Stakeholders

15. Kate
Usertesting.com for Branding
Skoodle Perooz Nimbus Iris Snap Right Now Breeze Bloom Boundless Launch Rendezvous

17. Flexible
Anytime
Anywhere
Holidays
Evenings
Weekends
No travel
Benefits

18. Can also be used for Mobile Phones & Tablets
Kirk

19. • Quick & easy
• Great for targeted tasks (“surgical
strike”)
• A great value
• For desktop or mobile
• Test anything from wireframes to
competitor sites
• Videos play in the browser
• Efficient - playback 1.5X or 2X
• Videos are downloadable
• Clips and annotations
• Tasks must be very clear – remember, users
are working on their own
• Users have to actually follow directions
• Un-moderated: No way to course-correct
• Not for time-consuming, complex tasks
• Selection bias: higher expertise user base
• You must watch all the videos
Usertesting.com – When To Use It
Kirk and
Kate

20. Video
Answers
User stats (technographics)
Video annotations
Video clips
Downloadable videos
spreadsheet
Review Live Results

21. Questions?

22. Thank you
kdoggett@vistaprint.com
klawrence@ebsco.com
@GR8UX
@bykatelawrence