Presentation on monitoring the web, including synthetic, UEUM, web analytics, interaction analysis. Given at www.meshconference.com/meshu on May 20, 2008

1. Watching websites

3. STARTUP 101

4. New idea End Execution Feedback Success? End Money left? Risk

5. Feedback, the old way.

6. Are people doing what we want?

7. Are we doing dumb stuff?

8. Do we understand our users? http://flickr.com/photos/ikhlasulamal/2443194039/

9. Is it easy and intuitive? http://flickr.com/photos/jmecelab/2323995433/

10. Slow, unmeasured trial & error.

11. The Internet lets us make mistakes faster.

13. New idea Execution Money left? End Feedback Success? New idea Execution Money left? Feedback Success? End Feedback Success? End Feedback Success? End

14. We do this by watching the web.

15. First: What business are you in?

16. Media

17. Transactions

18. Collaboration

19. Applications

20. Then: Know what we want to happen

25. Our victim…

26. Our eyes… Browser Data center Synthetic tester Analytics receiver Proxy Survey site Client-side interpreter Server Passive capture

27. The four big questions What did they do? Could they do it? Why did they do it? How did they do it?

28. The four big questions Web analytics What did they do? Could they do it? Why did they do it? How did they do it?

29. What matters in analytics Where did they come from? What attracts them best? Did they do what we wanted? Where did they drop out? What appealed most? Where did we send them? Acquisition Usage Referral

30. Data center Browser IT Analytics receiver Server

31. Data center Browser Client-side interpreter Server Analytics receiver

43. Page views, visits, visitors Top ten lists Demographics Technographics Top entry/exit pages Performance Capacity Security Path analysis Funnel reports A/B testing KPIs Dashboards Merchandising Segmentation SEO Campaign optimization Personas KPI alerts Multichannel aggregation Cost-shifting analysis Lifetime value Personalization Analytics-based content serving Process analysis Multichannel sales reporting Activity-based costing Balanced scorecards Strategic planning Predictive analytics Integrated user experience Adapted from Stephane Hamel and Bill Gassman IT-driven, “feel good” information, few decisions Business driven, working on metrics, accuracy and process Optimize the channel 330° view of customer (30° privacy) Strategic web business Level 1: Table stakes Level 2 : Fix the site Level 3 : Improve traffic Level 4: Complete view Level 5: MBA

45. The four big questions Web analytics User Experience Management What did they do? Could they do it? Why did they do it? How did they do it?

46. What matters in UEM? Could they get to the site? From everywhere? What regions were worst? What was their experience like? Did the app break? Was it fast enough? What things were slowest? Reachability Reliability Latency

47. The trivial web transaction Data center Browser TCP SYN (“let’s talk”) TCP SYN ACK (“Agreed: let’s talk”) TCP ACK (“OK, we’re talking) HTTP 200 OK (“Sure!”) TCP FIN (“Thanks! I’m done now.”) [index.html] (“Here it is!”) TCP FIN ACK (“You’re welcome. Have a nice day.”) HTTP GET / (“Can I have your home page?”) Server SSL (“Someone might be listening!”) SSL (“Here’s a decoder ring”) [img js css] (“Have this too!”) Bump, bump. (Renders furiously) (Thinks a bit)

49. Synthetic testing of key functions from around the Internet User monitoring of every transaction 2 complementary technologies Was it working ? Was it broken ?

50. Synthetic testing Synthetic testing of key functions from around the Internet User monitoring of every transaction Was it working ? Was it broken ?

51. Data center Browser Synthetic tester Synthetic tester … Server Synthetic tester

57. http://www.gomez.com/info_center/instant_test.php

62. User experience monitoring Synthetic testing of key functions from around the Internet User monitoring of every transaction Was it working? Was it broken?

63. Data center Browser Server Passive capture

72. The four big questions Web analytics Voice of the Customer User Experience Management What did they do? Could they do it? Why did they do it? How did they do it?

74. What matters in VoC? Why did they visit? Did they accomplish it? Why or why not? Motivation Success Reasons

75. Data center Browser Client-side interpreter Server Random selection Survey site

79. The four big questions Web analytics Voice of the Customer User Experience Management Web Interaction Analytics What did they do? Could they do it? Why did they do it? How did they do it?

80. What matters in WIA? Where did they go? How did they use the pages? What did they do wrong? Navigation Interaction Usability

81. Data center Browser Operator display The stage Mouse/key capture Server Analytics receiver

85. So how do I see what the user saw?

86. Data center Browser Display The stage Sample sessions’ stored pages Mouse/key & page capture Server Analytics receiver

90. How do I reduce client burden?

91. Data center Browser Display The stage All sessions stored pages Mouse/key capture Server Passive capture

94. (Just one more) Proxy communications

95. Browser Data center Proxy Client-side interpreter Server

101. Recap: Four big questions What did they do? Could they do it? Why did they do it? How did they do it?

103. NO SITE IS AN ISLAND

106. NON-HTML COMPONENTS

108. DYNAMIC PAGE NAMES

110. POP-UPS AND SITE DESIGN

113. PERFORMANCE WITH LOAD IN MIND

115. PRIVATE SITES

116. Privacy limits tools Data center Browser Synthetic tester Analytics receiver Proxy Survey site Server Private agents Passive capture OS agents

117. CLOUD COMPUTING PLATFORMS

118. Cloud limits server access Data center Browser Synthetic tester Analytics receiver Proxy Survey site Server Passive capture

119. AVERAGES LIE

120. 80 th percentile only spikes once for a legitimate slow-down (20% of users affected) Average varies wildly, making it hard to threshold properly or see a real slow-down. Setting a useful threshold on percentiles gives less false positives and more real alerts

121. GETTING WHAT YOU PAY FOR

123. STREAMING (COMET/BAYEUX, ADOBE)

124. How realtime web protocols work Data center Browser … HTTP 200 OK (“Sure!”) Here’s a channel to send me updates CSCO: $21 Subscribe to CSCO COMET server API in framework CSCO: $23 HTTP 200 OK (“Added!”) Subscribe to GOOG CSCO: $23 GOOG: $450 HTTP 200 OK (“Removed!”) Remove CSCO Got it

125. WATCHING BECOMES THE PROBLEM

127. Leftovers (other places to watch)

135. Join the conversation

136. Questions? (alistair at bitcurrent.com)