1. Social Mentions as Performance KPI
@jeroentjepkema
#SMC036, 1 October 2014

4. Key question:
Is the performance of my website hurting my business?

5. Waiting = Engagement

13. Mobile moments?

20. “Both offline and online consumers
associate long wait times with
poor customer service”

21. Click away slide Performance tolerance

22. 100
90
80
70
60
50
40
30
20
10
0
Bouncerate per pagetype/session
0 1 2 3 4 5 6 7 8 9 10
Bouncerate (%)
Page load time (sec.)
Median Campaign Product search
Real User Monitoring data collected by MeasureWorks for several eCommerce webshops, anonymized and grouped per pagetype

23. 100
90
80
70
60
50
40
30
20
10
0
Bouncerate per pagetype/session
0 1 2 3 4 5 6 7 8 9 10
Bouncerate (%)
Page load time (sec.)
Median Campaign Product search
Real User Monitoring data collected by MeasureWorks for several eCommerce webshops, anonymized and grouped per pagetype

24. Purpose vs. Context

25. Online users often lack context for delays...

26. ...and see no other option than to click away

27. Social = Performance??

28. First, understand your data

29. Every website has goals

31. Offer
Upselling
Purchase step 1
Purchase step 2
Conversion
Enrolment
Abondenment)
Organic Search
Campaigns
Ad Network
Visitor
Reach
Mailing,
alerts,
promotions
Disengagement)
€"
€"
€"
€"
€"
Impact)on)site)
€" Negative €" Positive
Transactional site

33. Targeted
embedded add
Enrolment
€"
Departure(
Impact(on(site(
€" Negative €" Positive
Media site
Add Network
Advertiser site
Visitor
€"
€"
€"

34. Analytics measures the movement
towards these goals

35. X
A simple online business model:
Marketing Conversion Optimization Revenue
New
visitors
Bounce
rate
Conversion
rate
Order
value
Growth
Loss
Time on
site
Pages per
visit
Number of
visits
Search
Tweets
Mentions
ADs seen

38. Why did customers drop off?
‣ Price
‣ Functional errors?
‣ Performance issues?

39. Why did customers drop off?
‣ Price
‣ Functional errors?
‣ Performance issues?
What’s the business impact?
‣ Lost customers?
‣ Revenue risked?
‣ In Euros?

40. Buy this book:

41. Web Analytics
(what did they
do on the site?)
Performance
(could they do what
they wanted to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)
Complete Web Monitoring
Competition
(what are they up
to?)
Social Media
(what were they
saying?)

42. Web Analytics
(what did they
do on the site?)
Competition
(what are they up
to?)
Performance
(could they do what
they wanted to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)
Social Media
(what were they
saying?)
Complete Web Monitoring
“Soft” data
“Hard” data

43. Context!

44. Web Analytics
(what did they
do on the site?)
Competition
(what are they up
to?)
Performance
(could they do what
they wanted to?)
VoC
(what were their
motivations?)
Usability
(how did they
interact with it?)
Complete Web Monitoring
“Soft” data
“Hard” data
Social Media
(what were they
saying?)

45. Social Optimization

46. Twinkle100: How fast are we?
(Twinkle100 represents the top100 eretailers and top30 travel companies
in the Netherlands in terms of annual revenue)

47. 1,49Mb
average
pagesize
Twinkle100 data collected with webpagetest.org with Chrome, IE11, Firefox & Safari (10Mbs down/1,5Mbs up)
4.59s
average
page load time
130
requests on
average
86%
slower than
3 sec.
Desktop Performance

48. 1,2Mb
average
pagesize
4.27s
via wifi
8.57s
via 3G
Mobile Performance
41% of
Twinkle100 have
optimized site
Twinkle100 data colelcted with webpagetest.org on both Android & iPhone (WiFi (10Mbs down/1,5Mbs up) & 3G (2000Kbps Down, 764Kbps Up))

49. Year over year performance (baseline = 2011)
-3%
25%
3% 5%
30%
17%
10%
Data collected with webpagetest.org on Crome, Firefox, IE11, Android & iPhone (WiFi (10Mbs down/1,5Mbs up) & 3G (2000Kbps Down, 764Kbps Up))
8%
4%
2012 2013 2014
2012 2013 2014
2012 2013 2014
0
Avg. Speed Avg. Size Avg. # Request

50. Click away slide Impact of performance?

51. Complaints Top 5 topics
21%
6%
41%
32%
Delivery Ordering UX Other
Downtime
Mobile
Speed
Login
Forms 5%
6%
21%
19%
36%
Research from MeasureWorks, Social and More & Obi4Wan. Social mentions collected about Twinkle100 webshops only between March 1 - August 31 2014

52. 40
30
20
10
0
0.00 2.00 4.00 6.00 8.00 10.00 12.00 14.00 16.00 18.00 20.00 22.00 24.00
Time of day (hours)
# of tweets)
Complaints per branche/day
Retail100
Travel30
Research from MeasureWorks, Social and More & Obi4Wan. Social mentions collected about Twinkle100 webshops only between March 1 - August 31 2014

54. Downtime
Mobile
Top 5 topics
Speed
Login
Forms 5%
6%
21%
19%
36%
Research from MeasureWorks, Social and More & Obi4Wan. Social mentions collected about Twinkle100 webshops only between March 1 - August 31 2014

55. Downtime
Mobile
Top 5 topics
Speed
Login
Forms 5%
6%
21%
19%
36%
18%
16%
28%
26%
12%
Mobile
Inchecken
Readiness
Contact
Vodafone
Abonnement
Research from MeasureWorks, Social and More & Obi4Wan. Social mentions collected about Twinkle100 webshops only between March 1 - August 31 2014

56. Downtime
Mobile
Top 5 topics
Speed
Login
Forms 5%
6%
21%
19%
36%
18%
16%
28%
26%
12%
Mobile
Check-in
Readiness
Contact
Vodafone
Subscription
33%
44%
23%
Forms
Empty fields
Mobile
Repeat task
Research from MeasureWorks, Social and More & Obi4Wan. Social mentions collected about Twinkle100 webshops only between March 1 - August 31 2014

57. Long term effect?

58. Performance Engagement Report
Page 1
Page 2
Page 3
Page 4
Page 11
Source:
Availability: Gomez
Sentiment: Twitter, Facebook
1200
400
0
-400
-800
-1200
Availability (%)
100
80
60
40
20
0
1200
800
Mentions (#)
Positive Messages Negative Messages Availability
Data collected from ING.nl Messages from Twitter & Facebook, collected with Coosto

59. Predicting Downtime

63. All systems green?

64. Or not?

65. 2 examples

66. Challenge: Can we predict downtime?

67. 60
45
30
15
0
0 5 10 15 20 25 30 35 40 45 50 55 60
# pageviews
Example downtime pattern
Min/Hour

68. 60
45
30
15
0
Regular operations
0 5 10 15 20 25 30 35 40 45 50 55 60
# pageviews
Example downtime pattern
Min/Hour

69. 60
45
30
15
0
Regular operations News
papers
0 5 10 15 20 25 30 35 40 45 50 55 60
# pageviews
Example downtime pattern
Min/Hour

70. 60
45
30
15
0
Regular operations News
papers Alerts?
0 5 10 15 20 25 30 35 40 45 50 55 60
# pageviews
Example downtime pattern
Min/Hour

71. 60
45
30
15
0
Regular operations News
papers Alerts? Downtime
0 5 10 15 20 25 30 35 40 45 50 55 60
# pageviews
Example downtime pattern
Min/Hour

72. Context ;-)

73. Min/Hour
# pageviews
60
45
30
15
0
Example downtime pattern
Regular operations News
papers Alerts? Downtime
0 5 10 15 20 25 30 35 40 45 50 55 60
This is where we need to be!

74. Example downtime pattern
2.
Social
Media
1.
Click
Behavior
Regular operations News
Min/Hour
# pageviews
60
45
30
15
0
papers Alerts? Downtime
0 5 10 15 20 25 30 35 40 45 50 55 60
This is where we need to be!
Min/Hour

75. Step 1: Profiling

77. Clickpath prediction

78. Step 2: Detection

79. How fast do I need to be?

80. Step 3: Creating context

81. Sentiment Behavior
60
45
+ 30
= Predicting downtime?
15
0
0 5 10 15 20 25 30 35 40 45 50 55 60

82. Test results
(Data derived from incident analysis of 3 websites, across a period of 4 months.
Data points matched (manually) over time period, per incident)

83. 60
45
30
15
0
0 5 10 15 20 25 30 35 40 45 50 55 60
# pageviews
Example downtime pattern
Min/Hour

84. Example downtime pattern
Change in click behavior
Time period in which we can detect a persistent
change in pattern per type of monitoring
60
45
30
15
0
Performance
alerting
Social Alerting
0 5 10 15 20 25 30 35 40 45 50 55 60

85. Your mileage may vary... for now, type of incident
determines predictive nature

86. Questions so far?

87. Real
time Day Week Month Quarter
Relevance for Application Life Cycle Keywords:
Alerting
Incident driven
MTTR
Keywords:
Business impact
Front End Optimization
Capacity Management
Data required
Keywords:
Error focused
Root Cause
Object driven information
Keywords:
User experience
Conversion metrics
Volume/Trends
Data cycle

88. Real
time Day Week Month Quarter
Relevance for Application Life Cycle Keywords:
Alerting
Incident driven
MTTR
Keywords:
Business impact
Front End Optimization
Capacity Management
Data required
Keywords:
Error focused
Root Cause
Object driven information
Keywords:
User experience
Conversion metrics
Volume/Trends
Data cycle Optimization, depending on
release cycle

89. Real
time Day Week Month Quarter
Relevance for Application Life Cycle Keywords:
Alerting
Incident driven
MTTR
Keywords:
Business impact
Front End Optimization
Capacity Management
Data required
Keywords:
Error focused
Root Cause
Object driven information
Keywords:
User experience
Conversion metrics
Volume/Trends
Keywords:
Real users
Contextual data
Trend, Velocity
Prevent
Data cycle Optimization, depending on
release cycle

90. Turning data into insights

91. Create patterns

92. Understand behavior

93. Contextual data

94. Listen!

95. Thanks! More questions?
M: jtjepkema@measureworks.nl
T: @jeroentjepkema
W: www.measureworks.nl