2024: Domino Containers - The Next Step. News from the Domino Container commu...
Boomerang at FOSS.IN/2010
1. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
Measuring the web with boomerang
Philip Tellis / philip@bluesmoon.info
FOSS.IN/2010 – 2010.12.15
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
2. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
$ finger philip
Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - paranoid - speedfreak
yahoo
http://bluesmoon.info/
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
3. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
4. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
5. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
6. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Less than 20% of page load time is something we can measure
and fix during development
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
7. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
It’s what we can’t control that bites us
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
8. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
browsers
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
9. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
plugins
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
10. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
OSes
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
11. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
viruses
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
12. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
antiviruses
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
13. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
microwaves
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
14. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
baby monitors
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
15. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
naughty neighbours
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
16. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
file shares
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
17. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
governments
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
18. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
rodents
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
19. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
Too many variations
Try simulating all that in the lab!
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
20. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
We need to measure real end-user performance
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
21. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
We need to measure real end-user performance from the real
end-user’s box
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
22. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
23. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
While this might work, it isn’t necessarily representative
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
24. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
What about JavaScript?
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
25. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
boomerang is...
A piece of javascript that you add to your web page where it
measures and beacons back to you, the end user’s perceived
performance of your page
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
26. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
How?
<script src="boomerang.js" type="text/javascript">
</script>
<script type="text/javascript">
BOOMR.init({
user_ip: "<user’s ip address>",
beacon_url: "http://mysite.com/beacon.php"
});
</script>
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
27. Introduction
Time
How does it work?
The adversary
Contributing to boomerang
Measure twice
Using boomerang
boomerang
Data
What does it do?
Measures user’s network throughput and latency to your
server
Measures the current page’s load time
Beacons these results back to your server
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
28. Introduction
Latency
How does it work?
Bandwidth/Throughput
Contributing to boomerang
Load time
Using boomerang
Accuracy
Data
How does boomerang work?
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
29. Introduction
Latency
How does it work?
Bandwidth/Throughput
Contributing to boomerang
Load time
Using boomerang
Accuracy
Data
Let’s take that one at a time
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
30. Introduction
Latency
How does it work?
Bandwidth/Throughput
Contributing to boomerang
Load time
Using boomerang
Accuracy
Data
Measuring latency
Download a 32 byte gif 10 times in sequence
Measure the time to download each
Discard the first measurement because it’s overpriced
Calculate the arithmetic mean, standard deviation and
margin of error of the rest
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
31. Introduction
Latency
How does it work?
Bandwidth/Throughput
Contributing to boomerang
Load time
Using boomerang
Accuracy
Data
Wait, did you say overpriced?
The first image might require a DNS lookup and TCP
handshake
Slow start is not an issue since 32 bytes fit in 1 packet
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
32. Introduction
Latency
How does it work?
Bandwidth/Throughput
Contributing to boomerang
Load time
Using boomerang
Accuracy
Data
Measuring throughput
After the latency test is done, we download progressively
larger images
Stop at the first image that times out
Redownload that image a few more times
Calculate the median, standard deviation and margin of
error of the largest images
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
33. Introduction
Latency
How does it work?
Bandwidth/Throughput
Contributing to boomerang
Load time
Using boomerang
Accuracy
Data
Measuring latency before throughput helps here
Those 10 latency images do a lot to widen the TCP
window size
The bandwidth images make much better use of available
bandwidth
The image we end with uses the most bandwidth
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
34. Introduction
Latency
How does it work?
Bandwidth/Throughput
Contributing to boomerang
Load time
Using boomerang
Accuracy
Data
How do we measure page load time?
In the onbeforeunload event, measure the time and
store it in a cookie
In the onload event, check the cookie, and measure the
difference with the current time
We also make sure that the page that set the cookie is the
referrer of the current page
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
35. Introduction
Latency
How does it work?
Bandwidth/Throughput
Contributing to boomerang
Load time
Using boomerang
Accuracy
Data
What? Two pages?
Yes, this needs two pages and cookies. If those aren’t
supported, we try to use the WebTiming API.
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
36. Introduction
Latency
How does it work?
Bandwidth/Throughput
Contributing to boomerang
Load time
Using boomerang
Accuracy
Data
How accurate is it?
Latency measurements are very accurate (±1%)
Bandwidth is to an order of magnitude. For bad
connections can be ±30%
Page load time sometimes has outliers, you need
post-filtering
The margin of error tells you how good your data is
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
37. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
Github
https://github.com/yahoo/boomerang/
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
38. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
Get the code
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
39. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
Get the code
git clone
git@github.com:yourname/boomerang.git
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
40. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
Submit code
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
41. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
boomerang.js
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
42. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
plugins
cp plugin.js yourplugin.js
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
43. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
The Makefile
make PLUGINS="plugin list" MINIFIER="/minifier/path"
make usage
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
44. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
Include it on your page
<script src="boomerang.js" type="text/javascript">
</script>
<script type="text/javascript">
BOOMR.init({
user_ip: "<user’s ip address>",
beacon_url: "http://mysite.com/beacon.php"
});
</script>
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
45. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
For most sites, that’s about it
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
46. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
You probably want to do more
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
47. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
Handle subdomains
Set the site_domain parameter:
BOOMR.init({
user_ip: "<user’s ip address>",
beacon_url: "http://mysite.com/beacon.php",
site_domain: "mysite.com"
});
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
48. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
This lets you measure transitions across *.mysite.com
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
49. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
Measure more than just load time
<html><head>
<script>var t_pagestart=new Date().getTime();</scrip
...
<script>var th=new Date().getTime();</script>
</head>
<body>
...
<script>var tj=new Date().getTime();</script>
...
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
50. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
...
<script src="boomerang.js"></script>
...
var te=new Date().getTime();
BOOMR.plugins.RT.setTimer("t_head", th-t_pagestart).
setTimer("t_body", te-th).
setTimer("t_js", te-tj);
</script></body></html>
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
51. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
This adds the t_head, t_body and t_js fields to the beacon
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
52. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
Loading dynamic content
BOOMR.init({
user_ip: "<user’s ip address>",
beacon_url: "http://mysite.com/beacon.php",
auto_run: false
});
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
53. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
Loading dynamic content
// Just before download starts
BOOMR.plugins.RT.startTimer("t_done");
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
54. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
Loading dynamic content
// Just after download finishes
BOOMR.plugins.RT.done();
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
55. Introduction Basic
How does it work? Multi-domain
Contributing to boomerang In-page timers
Using boomerang AJAX
Data
Much more
http://yahoo.github.com/boomerang/doc/howtos/
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
56. Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data
The beacon
GET request to the beacon URL (response ignored)
All parameters passed in the query string
Extra timers are passed in as a comma separated list in
t_other
before_beacon JavaScript event fired just before the
beacon is sent
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
57. Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data
What should we do with the data?
Sanity checking to:
Remove fake data
Remove abusive data
Maybe just rate limiting
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
58. Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data
What can we do with the data?
Statistical analysis to:
Remove outliers
Aggregate based on bandwidth blocks
Measure trends over time and correlate them with code
changes
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
59. Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data
Bandwidth blocks
Ref: Analysing Bandwidth & Latency – YUI Blog
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
60. Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data
Bandwidth blocks
Data points from some countries may require narrower bands
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
61. Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data
Geographic data
Looking at latency from different geographic locations can tell
you where your next mirror should be
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
62. Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data
ISPs
Grouping data by ISP can tell you who’s behaving badly
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
63. Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data
More data
Write plugins to get more performance data
We already have a DNS plugin
I’m thinking of an IPv6 v/s IPv4 plugin
What about a full WebTiming plugin?
Can we measure connection setup time?
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
64. Introduction
How does it work? shoulda
Contributing to boomerang coulda
Using boomerang woulda
Data
You decide
Once you have the data, you can do anything with it
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
65. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
Thank you
http://github.com/yahoo/boomerang
http://yahoo.github.com/boomerang/doc/
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
66. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
Photo credits
flickr.com/photos/21233184@N02/4389412851
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
67. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
Contact me
Philip Tellis
philip@bluesmoon.info
@bluesmoon
geek - paranoid - speedfreak
yahoo
http://bluesmoon.info/
slideshare.net/bluesmoon
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang
68. Introduction
How does it work?
Contributing to boomerang
Using boomerang
Data
References
github.com/yahoo/boomerang
More bandwidth doesn’t matter (much) – Mike Belshe
Analysing Bandwidth & Latency – YUI Blog
It’s the latency, stupid – Stuart Cheshire
The statistics of web performance
FOSS.IN/2010 – 2010.12.15 Measuring the web with boomerang