1. Guerrilla Usability Testing

2. Usability
"Good design is problem solving"
Jeffrey Veen
(Analytics,Typekit, author...)
Book: Psychology of everyday things
by Donald Norman

3. What’s Usability
• Usable, "easy to use". It's a very intuitive and simple
concept, yet many companies don't do it (officially)
• Like CT, but ABE one of the simplest Booking Engines.
• User Centered design. It has to be self-explained.
• Technology shouldn’t make us feel stupid, neither force
us to learn complicated stuff. “Don’t Make me Think”,
book by Steve Krug
• Jakob Nielsen : Focus on monetizing

4. Example 1 : Useit.com

5. Example 2 : Craiglists, another dull yet successful (usable) site

6. Example 3 : Amazon

7. Example 6 : Apple
and more; Google,Youtube...

8. Example 7 : ABE step 1. 3 simple steps (not just good for marketing)

9. ABE Step 2 grid layout. Not the most common layout, but it works.

10. ABE Step 3

12. Why things are unusable?
• Development focus on the machine
• It's hard to achieve
• Teams don't work integrated
• Design vs implementation
• Developer’s lack of perspective.
• like...

13. Unusable: Irishrail.ie : Complex, uses its own language...

14. Why things are usable?
• Early focus on users and task. Interactive Design
• Prototyping and wireframing
• Evaluation and measurement
• Research, Surveys, Observation
• like...

15. Usable : Dropbox

16. Development process stages
Analysis
Design
Evaluation
Implementation

17. Usability Testing
Usability testing is a technique
used to evaluate a product by
testing it on users...
Wikipedia
Usability testing generally involves
setting a series of tasks for people
to complete and noting any
problems they encounter
Andy Budd

18. Usability Testing Lab

19. Benefits on usability testing
For designers For site owners
• Help inform the design process • Improve customer satisfaction
and retention
• Uncover unknown or
unexpected issues • Increase conversion rates
• Fix problems early (easier) • Reduce maintenance, trainning
and support costs
• Test assumptions
• Reduce project risk
• Highlight unnecessary features
• Highlight unnecessary features
• Provide Objectivity solve
opinion battles • It’s so cheap you’d be stupid
not to
• Set baselines and measure
improvement
Andy Budd

20. Effective ways of testing
• Eye tracking. Heat maps
• A/B Testing
• Formal testing (made by a company, lab, focus groups...)
• Guerrilla style

21. Eye tracking: Heat map

22. A/B Testing : Used by Argus to implement Lightbox

23. A/B Testing : Used by Argus to implement Lightbox
“The results were encouraging: after 3 weeks of testing, the average landing page conversion rate
had gone from 2.70% to 3.33% (an average rise of 20% in the conversion rate)
Following the positive impact of the test, the light box functionality was rolled out across all PPC
landing pages, leading to marked increase in conversions (+19%)
This was then rolled out on to the rest of the site leading to an improvement of 5.99% in the
conversion rate for all traffic mediums”
(Thanks Dave)

24. Guerrilla Usability Testing
Guerrilla warfare is the
unconventional warfare and
combat in which a small group
of combatants use mobile
tactics in the form of ambushes
and raids to combat a larger and
less mobile formal army.

25. Guerrilla usability testing
• Get somebody from the street, and ask them to do simple task,
observe and record (using Camtasia or Silverback)
• Affordable (small rewards), informal. Everybody can help or bring
somebody
• Small group of people (5 get 85% issues, 12 get 99%)
• Moderator and “Think Aloud” protocol
• Precise task definition
• Using tools for metrics like analytics, surveys and feedbacks...

26. Pros and Cons of Guerrilla style
PROS CONS
• Help inform the design process • Improve customer satisfaction
and retention
• Uncover unknown or
unexpected issues • Increase conversion rates
• Fix problems early (easier) • Reduce maintenance, trainning
and support costs
• Test assumptions
• Reduce project risk
• Highlight unnecessary features
• Highlight unnecessary features
• Provide Objectivity solve
opinion battles • It’s so cheap you’d be stupid
not to
• Set baselines and measure
improvement

27. Silverback

28. How does it work

29. Gerrilla Testing
• We did it for the first time in September 2010 before push
live the new Step 3
• We found just a few issues; actually step 1: Magic locations
• For a moment we though, Is this really working?

30. Results

31. Gerrilla Testing
• Then we did it again, in the process of developing the map, this time
we did it earlier stage.
• Very different results, tough but positive feedback. They liked the idea.
• Things to work on and improve. But at least we knew what to focus
on.

32. 2nd Testing: The Map

33. Issues we found
• Pick up locations, most didn’t know they could put the address
• Zooms and controls
• Some didn’t see “I’m here”, confused the best price icon...
• Missing elements like scales
• Ignoring funcionalities, (Legend, Sort by, Refres)
• Didn’t fully understand which car was which flag in the map

34. Gerrilla Testing
Video

35. Work in progress...
Thanks
questions?