Bandwidth History of the Internet Midterm Questions How to tell if its a hardware or software problem? What makes the fan go? Benchmarking Cheating on Benchmarks Web Benchmarks Apache is very good!

2. Plan for Today
• Finish up Internet (from last class)
• Midterm Discussion
• Benchmarking
17 October 2013
University of Virginia cs4414
1

3. Measuring Networks
Latency
Time from sending a bit until it arrives
seconds (or seconds per geographic distance)
Bandwidth
Rate at which can you transmit
bits per second
17 October 2013
University of Virginia cs4414
2

4. $ traceroute -q 1 -w 30 www.busselton.wa.gov.au
traceroute to busselton.wa.gov.au (203.41.180.233), 64 hops max, 52 byte packets
1 dd-wrt (192.168.1.1) 11.156 ms
2 c-24-127-51-1.hsd1.va.comcast.net (24.127.51.1) 32.497 ms
3 te-2-3-ur01.charlville.va.richmond.comcast.net (68.85.226.149) 13.971 ms
4 xe-11-2-0-0-sur01.charlville.va.richmond.comcast.net (69.139.165.221) 12.312 ms
5 xe-4-1-2-0-ar02.charlvilleco.va.richmond.comcast.net (69.139.165.65) 12.395 ms
6 pos-1-2-0-0-cr01.ashburn.va.ibone.comcast.net (68.86.91.53) 25.624 ms
7 pos-3-10-0-0-cr01.56marietta.ga.ibone.comcast.net (68.86.86.221) 31.483 ms
8 pos-1-9-0-0-cr01.dallas.tx.ibone.comcast.net (68.86.87.233) 52.515 ms
9 he-0-12-0-0-cr01.losangeles.ca.ibone.comcast.net (68.86.86.117) 83.242 ms
10 as4637-cr01.losangeles.ca.ibone.comcast.net (75.149.228.222) 78.134 ms
11 i-0-2-0-11.tlot-core01.bi.telstraglobal.net (202.40.149.185) 86.131 ms
12 i-0-0-0-0.sydo-core01.bx.telstraglobal.net (202.84.140.5) 287.302 ms
13 tengige0-1-0-14.oxf-gw2.sydney.telstra.net (203.50.13.133) 300.060 ms
14 bundle-ether2.oxf-gw1.sydney.telstra.net (203.50.6.85) 274.270 ms
Does traceroute tell us anything 270.694 the
15 bundle-ether1.ken-core4.sydney.telstra.net (203.50.6.5)about ms
16 bundle-ether10.win-core1.melbourne.telstra.net (203.50.11.13) 275.252 ms
bandwidth between here an Busselton, Austrailia?
17 bundle-ether6.fli-core1.adelaide.telstra.net (203.50.11.90) 405.600 ms
18 bundle-ether5.wel-core3.perth.telstra.net (203.50.11.19) 411.510 ms
19 gigabitethernet0-1.wel13.perth.telstra.net (203.50.115.151) 406.044 ms
20 *
17 October 2013
University of Virginia cs4414
3

5. What Impacts Bandwidth?
IPv4: up to 99.96% efficient
LAN: Ethernet (97.6%
efficient for 12Kb packets)
WAN: PPP (99.9% efficient –
only 1-2 bytes overhead)
17 October 2013
University of Virginia cs4414
4

6. Below all this:
Physical Layer
17 October 2013
University of Virginia cs4414
5

7. Bandwidth
How much data can
you transfer in a given
amount of time?
17 October 2013
University of Virginia cs4414
6

8. Improving Bandwidth
• Faster transmission
– Train signalers to move semaphore flags faster
– Use something less physically demanding to transmit
• Bigger pipes
– Have multiple signalers transmit every other letter at the
same time
• Better encoding
– Figure out how to code more than 98 symbols with
semaphore signal
– Morse code (1840s)
17 October 2013
University of Virginia cs4414
7

9. Morse Code
Represent letters with series of
short and long electrical pulses
17 October 2013
University of Virginia cs4414
8

10. Circuit Switching
• Reserve a whole path through the network for
the whole message transmission
Paris
Bourges
Nantes
17 October 2013
Lyon
Toulon
Once you start a transmission,
know you will have use of the
network until it is finished. But,
wastes network resources.
University of Virginia cs4414
9

11. Packet Switching
• Use one link at a time
Paris
Lyon
Bourges
Toulon
Interleave messages – send
whenever the next link is free.
Nantes
17 October 2013
University of Virginia cs4414
10

12. internetwork
A collection of multiple networks connected
together, so messages can be transmitted
between nodes on different networks.
17 October 2013
University of Virginia cs4414
11

13. The First (international) internet
1800: Edelcrantz links
Sweden and Denmark
telegraph networks to
coordinate defense
17 October 2013
University of Virginia cs4414
12

14. End of First (international) internet
1801: British attach
Copenhagen; Sweden
doesn’t help Denmark;
network disconnected
17 October 2013
University of Virginia cs4414
13

15. The (capital-I) Internet
Packet Switching:
Leonard Kleinrock (UCLA) thinks
he did, Donald Davies and Paul
Baran, Edelcrantz’s signalling
network (1809)
Internet Protocol: Vint Cerf,
Bob Kahn
Vision, Funding (DARPA): J.C.R.
Licklider, Bob Taylor
Government: Al Gore
First politician to promote
Internet, 1986; act to connect
government networks to form
“Interagency Network”
17 October 2013
Vint Cerf (in the Rotunda, 2010)
University of Virginia cs4414
14

16. First Use of the Internet
October 1969: First packets on the ARPANet
from UCLA to Stanford. Starts to send
"LOGIN", but it crashes on the G.
How impressive is this compared to
communications event 3 months earlier?
17 October 2013
University of Virginia cs4414
15

17. 20 July 1969:
Live b/w video from the moon, transmitted
live to millions of televisions worldwide
17 October 2013
University of Virginia cs4414
16

18. Today’s Internet: Bandwidth
Ethernet:
up to 100 Mbits/sec
My office:
~50 M bits / sec
UVa Wireless:
57 M bits / sec
Cable modem at home:
~ 30 M bits / sec
17 October 2013
University of Virginia cs4414
17

19. 1 Petabyte = 1015 bytes
1 Million petabytes = 1 zettabyte
(NSA’s Utah facility is storing “Zbytes”?)
projections
17 October 2013
University of Virginia cs4414
18

20. Midterm Discussion
17 October 2013
University of Virginia cs4414
19

21. Easiest Way to Ace Midterm
4 people selected #9, but none
were bold enough to leave all the
other answers blank. (You should
only be “especially proud of”
your answer to #9 if you are
confident enough to leave all
other answers blank.)
17 October 2013
University of Virginia cs4414
20

22. How many processes should a
web browser create?
University of Virginia cs4414
several per “tab”
one per “tab”
several process
1 process
17 October 2013
21

23. How many threads should a
web browser create?
University of Virginia cs4414
several per “tab”
one per “tab”
several process
1 process
17 October 2013
22

24. How many processes should a web
browser developed in Rust create?
University of Virginia cs4414
several per “tab”
one per “tab”
several process
1 process
17 October 2013
23

25. How many tasks should a web browser
developed in Rust create?
University of Virginia cs4414
several per “tab”
one per “tab”
several process
1 process
17 October 2013
24

26. Your technically
unsophisticated (but
smart, rich, and
curious) uncle
complains that his
laptop is too noisy
since the fan is running
all the time. He
expects that as
someone taking upperlevel computing
courses at an elite,
public university, you
should be able to help
him. What do you do?
17 October 2013
University of Virginia cs4414
25

27. Hardware
problem
or
software
problem?
17 October 2013
University of Virginia cs4414
26

28. Hardware problem
or
software problem?
17 October 2013
University of Virginia cs4414
27

29. SOSP 2009
17 October 2013
University of Virginia cs4414
28

30. Uncle’s Fan Problems
Could it be
caused by
deadlock?
17 October 2013
University of Virginia cs4414
29

31. Fan Removes Heat
• CPU generates heat when it is
executing lots of instructions
• At least with my MBP, one
thread is not enough:
100% CPU usage – no fan
200% CPU usage – light fan
800% CPU usage – max fan
17 October 2013
University of Virginia cs4414
30

32. Performance
Benchmarking
17 October 2013
University of Virginia cs4414
31

33. Why Benchmark?
17 October 2013
University of Virginia cs4414
32

34. Developer Benchmarks
• Find bottlenecks: know what to spend time
optimizing
• Measure impact of changes to optimize
performance
• Predict what resources you will need to scale
service (“healthcare.gov”)
Goal is a benchmark that represents well the (anticipated)
actual usage
17 October 2013
University of Virginia cs4414
33

35. Micro vs. Macro Benchmarks
• Micro Benchmarks
– Test one specific operation
17 October 2013
University of Virginia cs4414
34

36. “Customer” Benchmarks
• Allow fair and accurate comparisons between
different solutions
• Standard accepted by all/many vendors
Goal is a benchmark that represents well the (anticipated) actual usage
for “typical” users
17 October 2013
University of Virginia cs4414
35

37. “Cheating” on Benchmarks
How could you “cheat” on a benchmark?
17 October 2013
University of Virginia cs4414
36

38. 1. On the Exynos 5410, Samsung was detecting the presence
of certain benchmarks and raising thermal limits (and
thus max GPU frequency) in order to gain an edge on
those benchmarks, and
2. On both Snapdragon 600 and Exynos 5410 SGS4 platforms,
Samsung was detecting the presence of certain
benchmarks and automatically driving CPU
voltage/frequency to their highest state right away. Also
on Snapdragon platforms, all cores are plugged in
immediately upon benchmark detect.
17 October 2013
University of Virginia cs4414
37

39. Everyone
(except Google
and NVidia) is
cheating!
Samsung is just
better at it than
anyone else.
17 October 2013
University of Virginia cs4414
38

40. Is It Possible to Prevent
Benchmark “Cheating”?
17 October 2013
University of Virginia cs4414
39

41. Benchmarking Zhtta
17 October 2013
University of Virginia cs4414
40

42. http://www.comlore.com/redist_files/httperf-www2007-slides.pdf
17 October 2013
University of Virginia cs4414
41

43. Example
httperf --hog --server=localhost --port=4414
--num-conns=1000 --rate=10
10 requests per second
Should you run httperf on the machine running your server?
17 October 2013
University of Virginia cs4414
42

44. Maximum connect burst length: 1
Total: connections 1000 requests 0 replies 0 test-duration 99.901 s
Connection rate: 10.0 conn/s (99.9 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 0.0 avg 0.0 max 0.0 median 0.0 stddev 0.0
Connection time [ms]: connect 0.1
Connection length [replies/conn]: 0.000
Request rate: 0.0 req/s (0.0 ms/req)
Request size [B]: 0.0
Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (19 samples)
Reply time [ms]: response 0.0 transfer 0.0
Reply size [B]: header 0.0 content 0.0 footer 0.0 (total 0.0)
Reply status: 1xx=0 2xx=0 3xx=0 4xx=0 5xx=0
CPU time [s]: user 27.52 system 72.40 (user 27.6% system 72.5% total 100.0%)
Net I/O: 0.0 KB/s (0.0*10^6 bps)
Errors: total 1000 client-timo 0 socket-timo 0 connrefused 1000 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
17 October 2013
University of Virginia cs4414
43

45. Macrobenchmarks
Need a workload that represents typical load
17 October 2013
University of Virginia cs4414
44

46. http://ita.ee.lbl.gov/html/traces.html
17 October 2013
University of Virginia cs4414
45

47. http://www.hpl.hp.com/techreports/1999/HPL-1999-35R1.pdf
17 October 2013
University of Virginia cs4414
46

48. 17 October 2013
University of Virginia cs4414
47

49. 1998
17 October 2013
University of Virginia cs4414
Image from http://www.hpl.hp.com/techreports/1999/HPL-1999-35R1.pdf
48

50. Macrobenchmarks
Need a workload that represents typical load
Synthetic Workload
17 October 2013
Replayed Workload
University of Virginia cs4414
49

51. Weilin’s Test Workload
> dd if=/dev/urandom of=5K.bin bs=5K count=1
> dd if=/dev/urandom of=5M.bin bs=5M count=1
> dd if=/dev/urandom of=10M.bin bs=10M /5K.bin
count=1
> dd if=/dev/urandom of=20M.bin bs=20M /index.htm
count=1
> dd if=/dev/urandom of=40M.bin bs=40M /5K.bin
count=1
/5K.bin
> dd if=/dev/urandom of=80M.bin bs=80M /80M.bin
count=1
> dd if=/dev/urandom of=512M.bin bs=512M count=1
/5K.bin
http://www.cs.virginia.edu/~wx4ed/cs4414/
ps3/zhtta-test-urls.txt
17 October 2013
University of Virginia cs4414
/5K.bin
/5K.bin
/5K.bin
/index.htm
/5K.bin
…
50

52. Testing Zhtta
> httperf --server localhost --port 4414 --rate 60
--num-conns 60 --wlog=y,./zhtta-test-urls.httperf
Total: connections 60 requests 60 replies 60 test-duration 32.588 s
Connection rate: 1.8 conn/s (543.1 ms/conn, <=46 concurrent connections)
Connection time [ms]: min 0.4 avg 1237.8 max 31954.9 median 563.5 stddev 4132.5
Connection time [ms]: connect 0.0
Connection length [replies/conn]: 1.000
Request rate: 1.8 req/s (543.1 ms/req)
Request size [B]: 67.0
Reply rate [replies/s]: min 0.0 avg 2.0 max 11.6 stddev 4.7 (6 samples)
Reply time [ms]: response 1215.7 transfer 22.1
Reply size [B]: header 71.0 content 16640936.0 footer 0.0 (total 16641007.0)
Reply status: 1xx=0 2xx=60 3xx=0 4xx=0 5xx=0
CPU time [s]: user 7.40 system 25.12 (user 22.7% system 77.1% total 99.8%)
Net I/O: 29920.6 KB/s (245.1*10^6 bps)
Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
17 October 2013
University of Virginia cs4414
51

53. Test Total Duration (seconds)
Zhtta with SRPT and caching
Zhtta with SRPT
Zhtta with FIFO
Zhttpto
Single-task server
0
10
20
30
40
Is our memory caching helping?
17 October 2013
University of Virginia cs4414
52

54. Test Duration (seconds)
Zhtta with SRPT and caching
Zhtta with SRPT
Zhtta with FIFO
Average Response Time (ms)
Zhttpto
Single-task server
Zhtta with SRPT and
caching
0
10
20
30
40
Zhtta with SRPT
Zhtta with FIFO
How good is this?
Zhttpto
Single-task server
0
17 October 2013
5000
10000
University of Virginia cs4414
15000
53

55. Possible to do Much Better!
Test Duration (seconds)
Apache2
Zhtta with SRPT and
caching
Zhtta with SRPT
Zhtta with FIFO
Zhttpto
Single-task server
0
17 October 2013
10
20
University of Virginia cs4414
30
40
54

56. Average Response Time (ms)
0.2 ms
Apache2
Zhtta with SRPT and
caching
Zhtta with SRPT
Zhtta with FIFO
Zhttpto
Single-task server
0
5000
10000
15000
httperf is measuring “Response Time” as latency to receive first byte!
17 October 2013
University of Virginia cs4414
55

57. Charge
PS3 is due less than one week from today! Still, no one has claimed OpenPiazza!
• You should definitely be able to show
improvements using Weilin’s benchmark (posted
with today’s notes)
• To prevent overly-targeted optimizing, we will
also test your servers with a different workload
• Demo will separately check correctness
– e.g., if a served file changes, should serve new file not
cached old one (within 10 seconds)
– To prevent “cheating”, we may look at your code
17 October 2013
University of Virginia cs4414
56