Nobody likes waiting… Especially (spoiled) online consumers, with all that content and competition just one click away. In this session Jeroen Tjepkema, founder of MeasureWorks, will explain the delicate balance between performance, usability and conversion. Based on real world examples and user panel research, this session will provide actionable insights in what's needed to create an user experiences that exceeds expectations. You'll learn how the mind of your user works, what keeps them clicking, what metrics to collect and how to measure performance and user experience.

2. @jeroentjepkema
Founder & CEO

3. Slides: bit.ly/MW-SBCmnfc13

4. Content
vs.
Experience

5. How we perceive experience....

6. How we perceive experience....
15%-20% worse than in reality

7. How we perceive experience....
15%-20% worse than in reality
Task completion has positive impact

8. How we perceive experience....
15%-20% worse than in reality
Task completion has positive impact
Slow downs have more impact

9. How we perceive experience....
15%-20% worse than in reality
Task completion has positive impact
Slow downs have more impact
Always compared to past experiences

10. A simple online business model:
Marketing
Conversion Optimization
New
visitors
Search
Tweets
Mentions
ADs seen
Conversion
rate
Growth
Number of
visits
Loss
Bounce
rate
Revenue
Pages per
visit
Time on
site
X
Order
value

11. Consumers visit 18-23 websites before
they finally convert
Source: TNS/Nipo

12. Performance influence cycle
Search/Orientation phase
1
Perception of experience
http://twinkle100.measureworks.nl

13. Performance influence cycle
Stimulate content/
conversion
4
Search/Orientation phase
5
1
Perception of experience
http://twinkle100.measureworks.nl
Delivered
experience
3
2
Your website

14. Fast
=
Engagement
@jeroentjepkema, MeasureWorks
@jeroentjepkema, MeasureWorks

15. 0
Source: Jakob Nielsen

16. 1
0,3
0
Interaction: Let’s conversate...
Instantaneous: I like it!
Source: Jakob Nielsen

17. 3
1
0,3
0
Mmm, shall I click away?
Interaction: Let’s conversate...
Instantaneous: I like it!
Source: Jakob Nielsen

18. 10
Only if the task/content is relevant
3
1
0,3
0
Mmm, shall I click away?
Interaction: Let’s conversate...
Instantaneous: I like it!
Source: Jakob Nielsen

21. 0:00 sec.
Conversation
starts

22. 0:00 sec.
Conversation
starts
2.5 sec.
First reply

23. 4.5 sec.
Finish
conversation
0:00 sec.
Conversation
starts
2.5 sec.
First reply

24. 4.5 sec.
Finish
conversation
0:00 sec.
Conversation
starts
Comfort zone
Tolerated
2.5 sec.
First reply
Frustrated

25. The holy grail in experience is
“flow state”

26. “”
Flow is an
“optimal experience”
that is
“intrinsically enjoyable”

27. Provide relevant
content to support
task completion

28. Deliver it fast,
focus on perception

29. http://www.nytimes.com/2012/08/19/opinion/sunday/why-waiting-in-line-is-torture.html?pagewanted=all&_r=0

30. http://answers.yahoo.com/question/index?qid=1005081200005

31. http://amzn.to/1biSi6C

32. That’s not my
website?

33. Over 60% of people that experienced a slow website ranked
design 2 points lower compared to a fast experience
Slow
Average
Fast
% of respondents that rated the actual website speed
100
75
50
25
0
1
2
3
Design score (1=bad - 5=beautiful)
4
5

34. Mobile
context?

35. Under time
Content
pressure
Mobile
context?

36. Under time
Content
pressure
Mobile
context?
(often) one
Usability
handed

37. Under time
Content
pressure
(often) one
Usability
handed
Mobile
context?
While
Speed
on the move

38. Under time
Content
pressure
(often) one
Usability
handed
Mobile
context?
While
Performance
on the move

39. M-commerce?

40. Experiment 1:
Mobile
Shopping

41. Mobile shopping experience setup....
100 mobile users
Perform 2 tasks:
‣Buy a book
‣Buy a T-shirt
Mobile browsers only
via 3G, no WiFi

42. Average experience rating
7
6
5
4
Fix it fast
3
2
1
1
2
3
4
5
Average expectation rating
Find a website
Search for a book
Order a book
6
7

43. #1:
Speed

45. Functional issues
reported with
Zalando
50
37,5
Round 1
25
45
42,1
12,5
15,4
0
Zalando
HM
20
7,7
V&D
24
3,9
Tom Tailor
4
Design
1. Buy a T-shirt
Speed
Mobile Readiness
Other
2. Feedback
40
30
Round 2
20
10
12
0
54
34
Zalando
Mobile Only research:
- Task completion: Only use smartphone to buy a book
- N = 100, users range from 20-65
15,4
HM
8
V&D
Tom Tailor
21
Design
18
Speed
6
Mobile Readiness
Other

46. Nexus - Android - 3G
Tested with webpagetest.org (3G traffic shaping, 2000Kbps Down, 1000 Kbps Up, 150ms latency)

47. Nexus - Android - 3G
Tested with webpagetest.org (3G traffic shaping, 2000Kbps Down, 1000 Kbps Up, 150ms latency)

48. #2:
Usability

49. Bol.com
Selexyz
De Boekerij
ECI
Bruna
Cosmox
Mobile
Readiness

50. 100
75
50
46
44
26
24
25
16
0
Bol.com Selexyz
Bruna
Other
1. Buy a book
Mobile Only research:
- Task completion: Only use smartphone to buy a book
- N = 100, users range from 20-65
Design
Speed
16
Mobile Readiness
2. Feedback bol.com
16
Other
Design
12
Speed
Mobile Readiness
Other
3. Feedback from different stores

51. Mobile
Readiness

52. -
One touch?
+

53. Login or
not?

54. Mobile
Search

55. Experiment 3:
Mobile
Banking

56. The mobile user experience…
50 users of ING, ABN
and Rabobank
Rate the user
experience
Select home insurance

57. There’s an app
for that?!

58. Mobile satisfaction compared to desktop
80%
74%
77%
Usability
Speed
23%
Design
Content
Satisfaction
Mobile Banking research:
- Focus on task completion
- N = 50, users range from 20-65

59. Mobile satisfaction compared to desktop
80%
74%
77%
Preference per type of task
77%
72%
38%
34%
23%
Design
Content
Usability
Satisfaction
Mobile Banking research:
- Focus on task completion
- N = 50, users range from 20-65
Speed
Check Balance
Transfer
3d Party payments View history
Mobile task
preference

60. Mobile satisfaction compared to desktop
Mobile versus Desktop frequency
Preference per type of task
Mobile
80%
74%
77%
77%
72%
38%
34%
23%
Design
Content
Usability
Satisfaction
Mobile Banking research:
- Focus on task completion
- N = 50, users range from 20-65
Speed
Check Balance
Transfer
3d Party payments View history
Mobile task
preference
Daily
Weekly
2-weekly
Change in
behavior
Monthly

61. Mobile
websites?

62. Desktop
Mobile
81%
71%
56%
62%
34%
42%
37%
22%
Design
Content
Usability
Speed
Design
Task satisfaction Desktop vs. Mobile
Mobile Banking research:
- Focus on task completion
- N = 50, users range from 20-65
Content
Usability
Speed

64. What impacts mobile (shopping) experience?

65. What impacts mobile (shopping) experience?
Day to day usage
Screen size vs. Content
Performance
Mobile browsers

66. Outrun your
competitors

67. Optimize all
your touch
points

68. Regular website
Mobile Web
Deliver the same experience to all your (mobile) users
Native Application

69. Mobile first
www.lukew.com

70. Deliver content accordingly
Prioritize your business goals
Mobile first
www.lukew.com

71. Design for
(mobile)
use cases

72. Focus on
task
completion

73. With
mobile
conditions
in mind

74. Optimized for the right screen

76. Native app
Mobile Web - Portrait
Mobile Web - Landscape

77. (Touch)
Screen

78. 49%
One device,
One hand
http://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobiledevices.php?goback=.gmp_72842.gde_72842_member_215909354

79. Golden
touch
http://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobiledevices.php?goback=.gmp_72842.gde_72842_member_215909354

80. Make it
(feel) fast!

81. Abandonment"Rate"Across"200+"Sites"/"177+"Million"Page"Views"
Over"2"weeks"/"All#Browsers#
30"
Abandonment"Rate"(%)"
25"
20"
Abandonment*Rate*,*
All*Browsers*
15"
10"
5"
0"
0"
1"
2"
3"
Source: Gomez real user monitoring
4"
5"
6"
7"
8"
9"
Page"Load"Time"Band"(sec.)"
10"
11"
12"
13"
14"
15"
It only
takes a
second...

82. Abandonment"Rate"Across"200+"Sites"/"177+"Million"Page"Views"
Abandonment"Rate"Across"200+"Sites"/"177+"Million"Page"Views"
Over"2"weeks"/"All#Browsers#vs.#iPhone#Safari#
Over"2"weeks"/"All#Browsers#
25"
Abandonment"Rate"(%)"
30"
25"
Abandonment"Rate"(%)"
30"
20"
20"
15"
15"
10"
10"
5"
5"
0"
Abandonment*Rate*,*
All*Browsers*
Abandonment*Rate*,*
All*Browsers*
Abandonment*Rate*,*
iPhone*Safari*
0"
0"
1"
1"
2"
2"
3"
4"
5"
6"
7"
8"
9" 10" 11" 12" 13" 14" 15"
0"
3"
4"
5"
6"
7"
8"
9" 10" 11" 12" 13" 14" 15"
Page"Load"Time"Band"(sec.)"
Source: Gomez real user monitoring
Page"Load"Time"Band"(sec.)"
Source: Gomez real user monitoring
..To click
away

83. Performance
Analytics

84. A simple online business model:
Marketing
Conversion Optimization
New
visitors
Search
Tweets
Mentions
ADs seen
Conversion
rate
Growth
Number of
visits
Loss
Bounce
rate
Revenue
Pages per
visit
Time on
site
X
Order
value

86. Why did customers drop off?
‣ Price
‣ Functional errors?
‣ Performance issues?

87. Why did customers drop off?
‣ Price
‣ Functional errors?
‣ Performance issues?
What’s the business impact?
‣ Lost customers?
‣ Revenue risked?
‣ In Euros?

88. We need
context!

89. Complete Web Monitoring

90. Web Analytics
Usability
Performance
(what did they do on
the site?)
(how did they
interact with it?)
(could they do what
they wanted to?)
Complete Web Monitoring
VoC
Social Media
Competition
(what were their
motivations?)
(what were they
saying?)
(what are they up
to?)

91. “Hard” data
Web Analytics
Usability
Performance
(what did they do on
the site?)
(how did they
interact with it?)
(could they do what
they wanted to?)
Complete Web Monitoring
VoC
Social Media
Competition
(what were their
motivations?)
(what were they
saying?)
(what are they up
to?)
“Soft” data

92. “Hard” data
Web Analytics
Usability
Performance
(what did they do on
the site?)
(how did they
interact with it?)
(could they do what
they wanted to?)
Complete Web Monitoring
VoC
Social Media
Competition
(what were their
motivations?)
(what were they
saying?)
(what are they up
to?)
“Soft” data

93. #1.
Collecting
Performance
Data

94. (CLOUD) DATA CENTER
INTERNAL USERS
CUSTOMERS
INTERNET
Third-party/
Cloud Services
Storage
Major
ISP
DB Servers Web Servers
Network
Mainframe
Middleware
Servers
App
Servers
Load
Balancers
This is what you control...
Local
ISP
Content
Mobile
Delivery
Networks Carriers
What you’re blamed for..

95. Measuring performance?

96. Measuring performance? Outside-in, from
the browser perspective...

97. Performance
Measurement
Toolkit

98. Benchmark &
Optimization

99. Webpagetest.org

100. Object level
!
Webpagetest.org
!
Optimization tips
!

101. Webpagetest.org

102. Ultimately, Real User benchmarking gives you periodic
insight in real usage scenario’s...

103. Uptime
Monitoring

104. Simulate business transactions

105. Via multiple devices & browsers

106. Used for error detection & Root Cause Analysis

107. Used for error detection & Root Cause Analysis

108. Ultimately, synthetic monitoring shows you if your
site’s working or not...

109. Real User
Monitoring

110. But, synthetic isn’t enough...

111. Synthetic heartbeat
Real Users
But, synthetic isn’t enough...

112. 1
2
Insert tag (.js file) into
(mobile) web pages
tag.js
Pages are requested
from browser/device
3
As pages execute,
tag collects
performance metrics
tag.js
tag.js
Browser RUM
tag.js
4
After onload tag send
detailed report for
further analysis

113. Relies on navigation timing API, custom variables can be added
Google Analytics

114. Soasta

115. Ultimately, Real User Monitoring shows you how many
users are affected by bad performance...

116. Performance
surveys

117. Expected Experience vs. Delivered Experience

118. Average experience rating
7
6
5
4
Fix it fast
3
2
1
1
2
3
4
5
Average expectation rating
Find a website
Search for a book
Order a book
6
7

119. Where UX meets Performance:
Study on Car Configurators (Audi vs. BMW)
http://bit.ly/MW-CC13
password: mob1l32013

120. Ultimately, performance surveys give you periodic insight
in task completion issues...
(20 people is enough to find 80% of common issues)

121. Summarized:

122. Uptime Monitoring
Real User Monitoring
Benchmark & Optimize
Performance surveys
Benefits
Benefits
Benefits
Benefits
Heartbeat, runs without traffic
Test specific customer journeys
Object level detail
Collect detailed alerts, including
root cause analysis
Desktop/Mobile Site
Pingdom
Watchmouse
Alertsite
Mobile Apps
Gomez
Keynote
Real usage information from
all users!!
Trending/Optimization
Business impact
Testing of user scenario’s
with real devices/bandwith
Optimization details
Competitive scan
Desktop/Mobile Site
Soasta
New Relic
Google Analytics
Desktop/Mobile Site
Webpagetest
Browserstack
Mobile Apps
Gomez
New Relic
Localytics
Google Analytics
Mobile Apps
Perfecto Mobile
Device Anywhere
Soasta
Performance Measurement toolkit
Soft data feedback
Abandonment optimization
Test before go live
Real User
Desktop/Mobile Site
Wufoo
Usabilla
Loop11
Crazyegg
Mobile Apps
Reflector
UX recorder
Magitest

123. #2.
Performance
Budget

124. What is your baseline:

125. What is your baseline:
A pre-defined set of metrics

126. What is your baseline:
A pre-defined set of metrics
that describes normal behavior

127. What is your baseline:
A pre-defined set of metrics
that describes normal behavior
in order to detect variancies

128. What is your baseline:
A pre-defined set of metrics
that describes normal behavior
in order to detect variancies
and to be comparable within historic context

129. Then define service levels & thresholds...

130. Purchasing a book,
must be completed (speed),
where every page loads under 4 sec.,
Customer journey
Metric: Speed
Target: Sec
using IE8 and higher,
User scenario
from any location in the Netherlands,
User locations
for 95% of all users,
every day between 6am and 12pm,
measured with Real User Monitoring.
Percentile
Window
Collection type
Source: Metrics 101, Velocityconf 2010

131. #3.
Performance
Analysis

132. Service levels
KPI’s
082
084
Performance (avg)
Loadtime.
Loadtime
Availability (Document.
(.fully.loaded.)
complete)
Competition
You
5
ww.homepage.nl
97,0%
2,20
3,50
10%
85%
/autoverzekeren
99,0%
2,50
3,30
20%
90%
ekeren/afsluiten
95,0%
2,30
3,60
15%
70%
95,0%
2,43
3,57
20%
67%
1
epage.nl/zoeken
94,0%
2,48
3,62
23%
59%
Branche
2
epage.nl/contact
Competitors
0
productoverzicht
93,0%
2,53
3,67
25%
92,0%
2,58
3,72
28%
44%
3
33
34
35
/ab8test/variant2
91,0%
2,63
3,77
30%
37%
/facebookplugins
90,0%
2,68
3,82
33%
29%
nl/muziekpagina
89,0%
2,73
3,87
35%
22%
homepage.nl/faq
88,0%
2,78
3,92
38%
14%
36
37
38
39
40
Week
52%
/ab8test/variant1
4
# 6
3,4 sec
Is your ranking
against
competitors
is the average
performance of your
competitors
40%
40% of your
compeitors are
faster than you

133. Competitive analysis
Your website
20
2
2000
1
1000
10
0
Pagesize (kb)
3
3000
Speed (sec)
URL’s
30
URL's 3rd
URL's
Speed (sec)
Page Size (KB)
0
33
34
35
36
37
38
39
40
Week #
33
34
35
36
37
38
39
40
Your site
Performance (FV)
3,3
2,4
2,2
2,8
3,3
2,9
3,1
3,2
URL's
67
68
65
69
80
72
69
73
URL's 3rd party
2
2
2
2
5
3
3
3
Page size (KB)
2000
1800
1850
2100
2400
2500
2500
2450
Your branche
Performance (FV)
2,3
2,0
2,5
2,8
2,3
2,4
2,2
2,5
URL's
55
59
56
62
59
60
59
61
URL's 3rd party
2
2
2
2
3
3
3
3
Page size (KB)
1900
1800
1850
1900
2050
1900
1890
1900
#30-35
#15-20
#10-15
#15-20
#50-55
#20-25
#30-35
#30-35
Competitive ranking

134. Performance Variation
9,26
8,98
7,79
7,79
7,23
7,29
7,24
7,24
6,67
4,97
5,4
5,40
4,80
4,61
3,39
3,38
3,08
3,08
2,97
1,89
Sunweb
D-reizen
Arke
Globe
Neckermann

135. Performance vs. Sentiment

136. 0!
dec;11!
nov;11!
okt;11!
sep;11!
aug;11!
jul;11!
jun;11!
mei;11!
apr;11!
200.000!
mrt;11!
feb;11!
Jan;11!
dec;10!
nov;10!
okt;10!
sep;10!
aug;10!
jul;10!
jun;10!
mei10!
apr110!
mrt;10!
feb;10!
jan;10!
dec;09!
nov;09!
okt;09!
sep;09!
aug;09!
jul;09!
jun;09!
Mei;09!
Web%traffic%%(Pageviews)!
Capacity Management
Max.%number%of%pageviews%per%hour%!
1.200.000!
1.000.000!
800.000!
Christmas/EOY%
600.000!
400.000!
Traffic!Realized!
Traffic!Forecast!
MAX!capacity!
Safety!capacity!

137. Revenue risked
Optimale flow
10.000 visitors
Speed (s)
Availability (%)
Bouncerate (%)
Conversie
Page 1
5,5
100
70
3000
Page 2
13,0
100
75
750
Page 3
2,0
100
60
300
Page 4
3,5
88,5
5
252
10.000 visitors
Actual flow
Speed (s)
Availability (%)
Bouncerate (%)
Conversie
Page 1
8,0
100
75
2500
Page 2
15,5
100
80
500
Page 3
2,0
100
60
200
Page 4
8,0
88,5
5
168

138. If anything else...

139. Build with
the end-user
in mind

140. Focus on a fast
experience
…without it,
NO time for
content

141. Deliver
before the
lights turn
green!

142. Reading material:
www.stevesouders.com
www.leananalytics.com
www.lukew.com

143. Thanks! More questions?
M: jtjepkema@measureworks.nl
T: @jeroentjepkema
W: www.measureworks.nl