Why is it that everyone knows the importance of frequent user
testing, yet hardly anyone does it? Because user testing often is time
consuming, complex and expensive. It probably doesn’t fit in your
development process and thus feels like extra work.
To feel reassured you tell yourself to test with users once you have
something working, or at the very end of the process. This is strange,
because everybody knows that changing your product late in the
process will increase costs exponentially.
We created a way so that user testing saves time, improves the
quality and doesn’t cost a lot of money. Team driven, pragmatic and
no extra resources needed.
The talk will show how, with only 2 hours every sprint, we focused on
creating better products faster. We would love to share our learnings
and simple DIY tools that let you start user testing with your current teams tomorrow!
4. Plus
The Netherlands
Appie (AH)
The Netherlands
Azbuka
Russia
Carrefour
France
Big C
Thailand
Alfa Mart
Indonesia
IceMobile: worldwide experience with a strong
focus on Loyalty & Retail
5. • Amsterdam based company (& office in Hong Kong, Toronto)
• 130 employees
• 35 nationalities
• Either a foodie, code king, design nerd, beer brewer or unicorn
IceMobile Amsterdam
6. Body text
• Bullet
Header text
Why?
How?
Customer Insights
Data Analysts & Data Scientist
UX Lab
UX Researchers / Data Storytellers
Who? What? When?
Where?
How many?
How often?
… What next?
RESEARCH TEAM
19. Transfer information effective
The team should learn most of the user test, not the UX researcher
Fit the user into the process
Do testing so that it is not ‘extra work’
Make results visible
Value should be clear and measurable
We wanted to find a way to:
21. Which practice proved its value?
Individuals and interactions
Working software
Customer collaboration
Responding to change
over
over
over
over
processes and tools
comprehensive documentation
contract negotiation
following a plan
Agile Manifesto
22. Source: Nielsen, Jakob, and Landauer, Thomas K.: "A mathematical model of the finding of usability problems," Proceedings of ACM INTERCHI'93 Conference (1993), pp. 206-213.
Usability Testing is an iterative process!
29. More evaluators = more issues identified
Source: Herttzum, M., Jacobsen, N. (2001). The evaluator’s effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer
Interaction, vol. 13, pp.421-443.
30. Ratio benefits to costs of # evaluators
4 = ideal
#
evaluators
Source: Herttzum, M., Jacobsen, N. (2001). The evaluator’s effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer
Interaction, vol. 13, pp.421-443.
33. Chapter title
How to keep track of the findings?
# user that
did this task
# user that had
this problem
time stamps
of this issue
on video
description
of issue
35. Pulse: iterative User Involvement
•Prioritize: test important part(s) most extensive
•Frequent: reliable results, also test improvements
•Empathize: learn what matters to users
•Optimize: learn to test
Allow responding to change
'I have not failed.
I've just found 10000 ways
that won't work.’
Sir Thomas Edison
36. Pulse UX TestingTraditional UX testing
Iterative UX testing: every 2 weeksUX testing (near) the end of a project
Continuous testingSingle test
Small sessionsBig session
Performed by the teams:
the team members learn most
Performed by researchers:
the researchers learn most
Changes early in the process;
thus less costly
Changes late in the process;
thus very costly
Rise in ‘first time right’ of featuresFew ‘first time right’ of features
120% end-result in 80% of the timeDifficult to measure success
37. What we achieved:
Transfer information effective
The team should learn most of the user test, not the UX researcher
Fit the user into the process
Do testing so that it is not ‘extra work’
Make results visible
Value should be clear and measurable