For a growing number of arts organizations, the web has become a key communications channel -- not just for sharing information, but for cultivating relationships with visitors. To what degree does your website deliver on its full potential? Does its visual and architectural design help or hinder visitors? What does it currently do well, and what specifically could improve? In this interactive workshop, you\'ll learn how usability testing can help to answer these questions. Created by Rober Barlow-Busch for the 2008 Technology in the Arts: Canada Conference.

1. How to improve your organization’s website
through usability testing
Technology in the Arts Canada 2008 • Robert Barlow-Busch
Photo by srboisvert on Flickr

2. Photo by E v a on Flickr

3. Today
Setting the stage
Planning a usability test
Conducting a usability test
Determining what you’ve learned
Goal: Prepare you to test your own website.

4. Setting the stage

5. Usability testing in 5 words:
Observe people using your website

6. Usability testing in 21 words:
Observe people using your website
Identify issues…
By watching representative visitors…
Complete common tasks on your website.
Then fix the issues!

7. But why?
Make your website more useful to people.
Make your website more easy to use by people.
Make people want to visit — your website or your organization.

8. From informal, guerilla testing… …to the icing on the cake.
Photo by gorriti on Flickr Photo by chotda on Flickr

9. Welcome to the family!

10. Planning a usability test

11. Whom should you select as participants?
People representative of your actual (or desired) visitors.
Not yourself.
Not your designer(s).
Not people in your organization.

12. How many participants?
1? 8? 18? 80?
A gazillion?

13. Finding participants
Where to find them:
• A recruiting firm
• Your personal and professional network
• Current visitors (online or bricks ‘n mortar)
How to entice them:
• Don’t ask for help —offer them a fun opportunity!
• Provide incentives such as $$ or gift certificates

14. Where to conduct your usability test?

15. Roles on the testing team
Facilitator (mandatory)
• Interacts with the participant and runs the test.
Scribe (optional)
• Takes comprehensive notes.
Equipment jockey (optional)
• Sets up and monitors technology such as recording equipment.
Observer (optional)
• Quietly watches the test
• Usually a stakeholder from your organization.

16. Determining what to test
Option 1: Formulate tasks in advance
•
• Ensures you test specific parts of the website
• Appropriate when you know what people want to do on the site
Option 2: Interview participants and create ad-hoc tasks:
•
• Ensures you test the website for realistic activities
• Appropriate when you’d like to know what people want to do on
the site

17. Writing task instructions
Use the participant’s language.
Don’t inadvertently provide hints in your wording.
• “See if the museum has a guest policy for children’s strollers”
• “Find out if you’re allowed to bring a stroller to the museum”
Tasks can be very specific…
• “Can you visit the gallery this Saturday at 5:30 pm?”
…or rather general
• “Find out if this organization offers any services that interest you.”

18. What websites shall we test today?
Some ideas: Suggestions or volunteers?
• Technology in the Arts (.org) • Site should be of reasonable
• Royal Ontario Museum size/richness
• Waterloo Children’s Museum • Bonus: knowing business
• CBC Radio: Ideas goals so we can create tasks
• Canada Council for the Arts to test them

19. Exercise: Planning a usability test
1. Select a website to test.
2. Identify five tasks and write instructions for each.

20. Conducting a usability test

21. A common schedule
Orientation: 15 minutes
•
Tasks:
• 1 hour
Debriefing: 15 minutes
•
TOTAL:
• 90 minutes
Photo by Mrs. Maze on Flickr

22. Orienting your participant
Explain the purpose of your test
•
• “You’ll be using our website today to help us figure out where it needs
improvement.”
• “We’re not testing you. You’re helping us test the product.”
Interview them to:
•
• Confirm they meet your recruiting criteria (expect surprises sometimes)
• Gather information for planning ad-hoc tasks
• Establish rapport

23. Facilitating the test
Ask participants to “think aloud” while they perform tasks
•
• Can be very helpful if you demonstrate first
Probe or prompt for their thoughts if necessary
•
• Does their body language reveal or suggest anything?
• “What’s going through your mind now…?”
Avoid showing approval or disapproval.
•

24. Testing with younger audiences
Plan on less hands-on testing time
• More time needed to put them at ease
• Kids can find it hard to focus for long periods
Invite parents to accompany them
• But advise parents not to influence them; seat
them out of sight?
Expect more requests for help
• “What do you think you’d do here?... Any
guesses?... How about one last try…?”
Make it a playful experience!
Photo by Hey Paul on Flickr

25. Taking notes
DO record observations: what you see and hear
• E.g., Suzie never scrolled down the page
DON’T record inferences: why you think something happened
• E.g., Suzie doesn’t like to scroll
Take note of “sound bites”
• “It’s like trying to drive a car with the dashboard ripped out!”

26. Debriefing
Probe for qualitative feelings about the website
• What are the two things you most liked about the site? Disliked?
Collect quantitative data if desired. E.g., On a scale of 1–5:
• How successful were you in accomplishing today’s tasks?
• How would you rate the site’s visual design?
• How confident did you feel about where you were in the site at all
times?

27. Photo collage:
How I feel about your
organization…

28. Photo collage:
...and how I feel about
your website.

29. vs.

30. Exercise: Conducting a usability test
You have 15 minutes to conduct each test.
Tips for each role:
• Facilitator: Explain tasks, prompt to think aloud, deflect questions
• Scribe: Record positives, negatives, and good “sound bites”
• Participant: Do your best to role-play if you’re not the target audience
Rules:
• Do not be the participant if it’s your website
• When we switch, take on a role you haven’t tried yet

31. Determining what you’ve learned

32. Identify the highs and lows
Review your notes from each test
•
• Catalog both usability issues and what went well
Look for patterns across all your tests
•
• Try “affinity diagramming”
Prioritize the issues:
•
• Showstopper: Participant couldn’t proceed
• Serious: Caused major confusion or loss of confidence
• Improvement: A minor annoyance

33. Should you recommend fixes?
Photo by zerega on Flickr

34. Report options
Do nothing!
List highlights in an email.
Write a document outlining the methodology, observations,
and results.
Create a multimedia extravaganza.

35. Exercise: Determining what you’ve learned
Form groups of people who were Scribes for each site.
Each of you provide 1 positive finding, 1 issue, and 1 sound bite.
As a group, make one recommendation for a high-impact fix.

36. Q&A

37. Thank you!
Robert Barlow-Busch
Director of Product Design
Primal Fusion Inc.
bbb@primalfusion.com
www.primalfusion.com
Blog: www.chopsticker.com