My talk at the LSPE meetup in Jan 2013. http://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/92123162/

1. • Philip Tellis
• @bluesmoon
• philip@bluesmoon.info
• SOASTA
• boomerang
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 1

2. Measuring the Performance Experienced by
Real Users
Philip Tellis / philip@bluesmoon.info
LSPE Meetup / 2013-01-17
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 2

3. Real Users
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

4. Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from experience
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

5. Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from experience
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

6. Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from experience
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

7. Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from experience
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

8. Real Users
• Get delighted
• Get frustrated
• Have credit cards
• ... that they are attached to
• Form an impression from experience
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 3

9. What can we measure?
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 4

10. Anything that the user does!
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 5

11. Anything that the user does!
(in the browser)
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 5

12. But we really care about performance here. . .
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 6

13. So let’s talk about How
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 7

14. Modern Browsers – no problem
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 8

15. Modern Browsers – no problem
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 9

16. Real Users don’t always use the latest browsers.
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 10

17. What about older browsers?
1 Get a timestamp before they leave a page
2 Save it in a cookie
3 Compare with timestamp when the page loads
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 11

18. What about older browsers?
1 Get a timestamp before they leave a page
2 Save it in a cookie
3 Compare with timestamp when the page loads
Then there’s some magic to handle edge-cases
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 11

19. The magic. . . ...
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 12

20. But it doesn’t need to be. . .
http://lognormal.github.com/boomerang/doc/
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 13

21. boomerang
• BSD licensed
• Measure load time, bandwidth, latency
• DNS lookup time, NavTiming, Connection Type
• Page Parameters
• And anything else you can write a plugin for
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 14

22. And it beacons all of this back to a predefined URL
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 15

23. So what then?
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 16

24. Sparse Distribution
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 17

25. Log-normal distribution
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 18

26. Bimodal distribution
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 19

27. Distributions
• Sparse distribution suggests that you don’t have enough
data points
• Log-normal distribution is typical
• Bi-modal distribution suggests two (or more) distributions
combined
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 20

28. Distributions are interesting, but how does it affect
engagement?
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 21

29. So we compared load time to bounce rate
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 22

30. And we borrowed a biological term called LD50
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 23

31. LD50
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 24

32. This helps you fix only the stuff that’s hurting you.
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 25

33. Did you know that IE7 seems to be faster than IE8?
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 26

34. But that’s mostly IE9 users browsing in IE7
compatibility mode.
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 27

35. Did you know that users bounce if a page is too
fast?
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 28

36. But that’s mostly because error pages are the fastest.
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 29

37. Did you know that user patience wrt performance is
a cultural thing?
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 30

38. It’s tied both to where a user is from, and also the
type of service.
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 31

39. Demo?
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 32

40. Summary
1 Measure the load time experienced by real users
2 Correlate it with other things that user did
3 Profit
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 33

41. Summary
1 Measure the load time experienced by real users
2 Correlate it with other things that user did
3 Profit
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 33

42. Summary
1 Measure the load time experienced by real users
2 Correlate it with other things that user did
3 Profit
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 33

43. Thank You!
Questions?
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 34

44. • Philip Tellis
• @bluesmoon
• philip@bluesmoon.info
• www.SOASTA.com
• boomerang
• LogNormal Blog
LSPE Meetup / 2013-01-17 Measuring the Performance Experienced by Real Users 35