Presented during Customer Week in Chicago, 2011. Explores ways to make sure your online Customer Self Service Solution is effective.

1. Optimize Customer
Self-Service by
Combining Metrics
with Customer Feedback
Martin Krohne
Senior Manager User Experience, AutoTrader.com

2. what?
2

3. a l l n e w t i t l e :
how to make sure
your self-service solution
actually works
3

4. every time
I think I know my customers,
they surprise me
martin krohne - when asked for a quote for this conference - Jun. 2011
4

5. because I am not my customer
5

6. you are
6

7. why performance metrics
don’t tell you what
you need to know
7

8. performance metrics
tell you whatmost of your users did
8

9. performance metrics
do not tell you why any users did that
9

10. why surveys
don’t tell you enough
10

11. surveys are only as good
as the questions you ask
11

12. when you assume
that you know what to ask
you might not learn enough
12

13. multiple choice questions
limit discovery to what you already think
the answers are
13

14. how to find out what’s
really going on
14

15. qualitative research
can give you a better idea
of where the problems really are
15

16. focus groups
can uncover differences in opinion
and identify social pressures
16

17. usability testing
can uncover hidden problems
and offer opportunities for probing
17

18. direct observation
can uncover issues
you would never discover otherwise
18

19. one-on-one interviews
can help you determine pain points
and frustrations with your service
19

20. testing,
observing
and interviewing
tips
20

21. watch what they do
to learn what the real problem is
when using your self-service solution
21

22. ask them what it was
that they wanted to do in the first place
22

23. ask them why
they left without completing their task
23

24. but don’t ask them
how to fix it
because they don’t know what you know
24

25. clarify their pain points
it is up to you to come up with a solution
because you know what is possible
25

26. a better way
to use surveys
26

27. after you know
why your customers
are struggling
27

28. take these insights
and build the questions you need to ask
to learn which issues are most important
28

29. use the survey results
to confirm what you’ve learned so far
and to prioritize the most important issues
29

30. pick the biggest issue(s)
don’t try to solve everything at once
look for the biggest impact

31. begin
developing
solutions
31

32. clearly state the problem
what did customers struggle with?
what task were they trying to complete?
what was their biggest pain point?
32

33. identify success metrics
that clearly define a successful solution
so that others can understand it
33

34. develop multiple solutions
explore and exhaust all possible approaches
and then select at least three
completely different solutions
34

35. only your customers
can tell you
if you guessed right
35

36. test early and often
don’t wait for the final solution
to be complete - you want to know what fails
as early as possible
36

37. test acceptance
put your solutions in front of some customers
to learn what works… and what doesn’t
37

38. rethink your approach
look for other ways to solve the problem
while avoiding past missteps
38

39. test your new solution(s)
with your customers
find out what you really fixed and
learn what doesn’t work

40. evaluate results
ensure you addressed the most critical issue
without breaking anything else important
40

41. do it over and over
keep re-designing and testing until you’ve
solved the big problems (not all of them)
41

42. why keep testing?
when you fix a big problem,
a new one is likely to appear
42

43. this is normal
sometimes a big problem
masks several smaller ones
43

44. the final test
44

45. test your best alternative
in a live setting against your current
self-service product
45

46. reality check
revisit your success metrics so you know
whether or not this is an improvement
46

47. stop
when you have an improvement
not the most perfect solution
47

48. launch and start over
because...
48

49. customer expectations
will continue to change and you need to know
which changes are most important to address
49

50. what did i just say?
50

51. you are not your customer
you need to find out how they think
and why they are struggling

52. data is not the answer
performance metrics will not tell you why
customers are having trouble
52

53. surveys
are not the only
or the best approach
you may only learn
what you already think you know
53

54. use qualitative research
to get a broader sense of the issues and to
drive future quantitative research
54

55. use surveys later
to verify what you learned in qualitative
and to prioritize those issues

56. develop
multiple
solutions
you will quickly learn what is feasible
and are likely to discover innovative ideas
56

57. test early and often
a small group of customers can help you
quickly determine which solutions might work
57

58. test your best alternative
use clearly defined metrics that support solving
the problem(s) you’ve observed and verified
58

59. stop
when you have significant improvement,
not the most perfect solution
to your customer’s problem
59

60. launch and start over
big problems mask smaller ones
and customer expectations
continue to grow
60

61. any questions?
61

62. 62