This document discusses the evolution of quantifying user experience over time. It traces the development of tools and methods for observing user behavior from pen and paper tests in the 1980s to modern remote usability testing platforms in 2009. Both qualitative and quantitative methods are examined, ranging from conceptual attitude-based approaches to concrete behavior-based metrics. The document advocates combining both qualitative and quantitative data sources to develop a robust understanding of user experience.

1. Quantifying UX
The lab versus the real world

2. How did I end up here?

3. 2003-2010
Tilburg University
Design of Business Communication

4. 2000-2010
Design, communication, research, and consulting

5. 2009-now
cc adactio

6. !"#$%&#"'$!()%#*+,+!-

7. ‘Plus-minus method for text evaluation’

8. + -

11. </commercial>

12. eighties

15. Observing users.

16. Pen & paper

17. Stopwatch.

19. Nothing changed?

22. How do we observe behavior?

23. 1995 Web ‘analytics’

24. 2005 Google Analytics

26. 2007 Clicktale, CrazyEgg, Kampyle

28. 2009 Remote Usability Year

29. 2009 UserTesting

31. 2009 FeedbackArmy

33. 2009 UserZoom

35. 2009 OpenHallway

37. 2009 Userfly

39. 2009 OptimalSort

40. 2009 ChalkMark

41. 2009 Treejack

43. 2009 Userlytics

45. 2009 Loop11

47. 2009 Websort

49. 2009 Ethnio

51. 2009 TrymyUI

53. 2009 Usabilla

55. More?
Great overview: http://remoteusability.com

57. Qualitative
Conceptual
Concrete
Quantitative

58. Qualitative
UserTesting
TryMyUI
Userlytics
OpenHallway
Conceptual
Concrete
Kampyle Usabilla
Loop11
Websort
Userfly Treejack
Optimalsort
Chalkmark
Clicktale
CrazyEgg
Google Analytics
Quantitative

59. Source: Nate Bolt, Bolt | Peters / Ethnio - http://boltpeters.com

60. Source: Nate Bolt, Bolt | Peters / Ethnio - http://boltpeters.com

61. Behavior vs Attitude

62. Behavior vs Attitude

63. Behavior vs Attitude
Source: the narrow minded usability consultant.

64. Usability
effectiveness efficiency

65. Usability
effectiveness efficiency satisfaction

66. behavior = objective
effectiveness efficiency satisfaction

67. behavior = objective attitude = subjective
effectiveness efficiency satisfaction

69. User Experience
Findable Desirable Accessible
Valuable Credible
Usable

71. User Experience
Findable Desirable Accessible
Valuable Credible
Usable

72. ‘self reported data’ hoax
Some background: http://www.creative-wisdom.com/teaching/WBI/memory.shtml

74. Soft data

75. Hard data

77. Behavior:
Conversion rates, task
performance, time on
Hardcore task, bounce rate, error
rates, click-through rate,
shares, bookmarks, etc

78. Attitude:
Satisfaction, ratings,
credibility, sentiment
analysis, feedback, trust,
Shell like / dislike, etc
Soft

79. So, what’s next?

80. Booming.

81. Build your toolbox.

86. Mobile?

87. Commercial break
The most touch friendly survey
!"#$%&#"'$!()%#*+,+!-

88. Data.

89. Data.

90. Data.

91. Dashboards

94. Thanks for your attention!
E-mail: paul@usabilla.com
Twitter: @pveugen
Website: usabilla.com
Paul Veugen
Usabilla.com
!"#$%&#"'$!()%#*+,+!-