From request to response, this session will dive deep into identifying, measuring and reducing latency in your stack for lightning fast applications that keep your customers smiling. Learn how to architect for performance and fine tune your apps for speed with real world examples using the services you love: EC2, RDS, DynamoDB and S3.

1. Latency’s Worst Nightmare:
Performance Tuning Tips and Tricks
Matt Wood
Principal Data Scientist
@mza

2. Hello.

3. Let’s talk about performance...

4. Productivity and response time
A. J. Thadhani, IBM Systems Journal 20 (4), 1981

5. Page load time and average daily searches per user
http://www.webperformancetoday.com/2013/04/10/cloud-connect-2013-web-acceleration-and-front-end-optimization-slides/

6. Page load delay and business metrics
http://www.webperformancetoday.com/2013/04/10/cloud-connect-2013-web-acceleration-and-front-end-optimization-slides/

7. 2.2s 15.4%
reduction in page load time increase in conversion rate
https://blog.mozilla.org/metrics/2010/04/05/firefox-page-load-speed-%E2%80%93-part-ii/

8. “Speed is more than a feature.”
@fredwilson

9. Let’s talk about a web request...

10. %CPU
kbps
Initial connection SSL negotiation Time to first byte Content download

11. Remote
%CPU
kbps
Initial connection SSL negotiation Time to first byte Content download

12. Remote Browser
%CPU
kbps
Initial connection SSL negotiation Time to first byte Content download

13. Remote Browser
%CPU
kbps
Initial connection SSL negotiation Time to first byte Content download

14. http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/

17. Let’s talk about a web request...

18. Web server
Application logic
Database

19. Web server
Application logic
Database

20. Web server
Application logic
Database

21. Web server
Application logic
Database

22. Web server
Application logic
Database

23. Web server
Application logic
Database

24. Web server
Application logic
Database

25. Web server
Application logic
Database

26. 7
1
2
Web server
3
Application logic
4
Database
5 6

27. 7
1
2
Web server
3
Application logic
4
Database
5 6

28. TTFB
%CPU
kbps
Initial connection SSL negotiation Time to first byte Content download

29. Network latency Download + negotiation time
%CPU
kbps
Initial connection SSL negotiation Time to first byte Content download

31. NYC London Sydney
Variable TTFB based on geographic location.

32. Reduce internet induced latency.
CloudFront content delivery network.
Lower latency. Faster downloads.

33. Reduce internet induced latency.
CloudFront content delivery network.
Lower latency. Faster downloads.

34. CloudFront Edge Locations

35. Static and dynamic content
Cache dynamic pages (search results).
Use query strings or cookie for cache keys.
Network and Path optimizations accelerate
even unique content.

36. 7
Content delivery
1
2
Web server
3
Application logic
4
Database
5 6

37. 7
Content delivery
1
2
Web server
3
Application logic
4
Database
5 6

38. Web server
Application server

39. Web server
Application server

40. Application server

41. Thread safe?
Application state? Application server

42. Web server
Application server
Database

43. Unit of scale
Web server
Application server

44. Load balancer
Web server
Application server

45. Load balancer
Web server Web server Web server
Application server Application server Application server

46. Load balancer
Web server Web server Web server
Application server Application server Application server

47. Load balancer
Web server Web server Web server
Application server Application server Application server

48. Decouple your service tiers
Separation of concerns.
Easier to manage.
Drives higher availability.

49. Build for horizontal scale
Decrease request contention.
Reduce capacity planning headaches.
Requires a stateless application architecture.

50. Small things, loosely coupled.
Do one thing, and do it well. The Unix Way.
Take a look at the Unicorn and Rainbows approach.
Asynchronous be default (where possible).

51. Reduce response time.
Concurrency limits can reappear quickly.
Limit impact on performance through rapid scaling.

52. Load balancer

53. Load balancer

54. Load balancer

55. Fast booting with EBS-backed instances.
Linux is faster to boot than Windows.
EBS-backed instances are faster than S3 backed.

56. Pre-baked EBS-backed AMIs.
Each code deployment creates a new AMI.
AMI is the unit of deployment.

57. Automate response with Auto Scaling.
Set operational thresholds.
Faster than manual response (especially at 3am).

58. Time-based response with OpsWorks.
Set operational times.
‘Follow the sun’ response.

59. Pre-emptive scaling.
Contact your account manager, or
get in touch via a Premium Support ticket.

60. Reserve capacity.
Lower costs.
Guaranteed availability.

61. 7
Content delivery
1
Horizontal scale
2
3
4
Database
5 6

62. 7
Content delivery
1
Horizontal scale
2
3
4
Database
5 6

63. A range of instance types
Full spectrum of price/performance options.

64. Choosing the right instance type.
Application specific.
Benchmark.
CloudWatch metrics.

65. Benchmark on business metrics.
Relate application metrics to business metrics.
Customers supported/instance.
Photos processed/dollar.

66. The Canary in the Coal Mine
Standardize on 64-bit AMIs.
Deploy across instance types.
Evaluate new instance types with real traffic.

70. 7
Content delivery
1
Horizontal scale
2
Instance selection
3
4
Database
5 6

71. 7
Content delivery
1
Horizontal scale
2
Instance selection
3
4
Database
5 6

72. Interface with the data store.
Faster if you don’t have to go to disk.
Increased concurrency.

73. Caching.
Store query results in memory.
Writes go to disk.

74. Amazon ElastiCache.
Deploy, operate and scale in-memory caches.

75. Managing state.
Transient data only.
Web server state, high score tables, etc.
Time consuming task results
(many to many query results).

76. Best practices.
Assume cold cache latency in application architecture.
Set appropriate time-to-live.
Batch requests rather than sequential single.
Architect for cache failure.

77. 7
Content delivery
1
Horizontal scale
2
Instance selection
3
Caching
4
Database
5 6

78. 7
Content delivery
1
Horizontal scale
2
Instance selection
3
Caching
4
Database
5 6

79. Accelerating reads.
Vertical scale vs horizontal scale.

80. Horizontal scale.
Add additional DB resources for scale.
Scale out for reads.
Shard for reads and writes.

81. Vertical scale.
More resources for a single DB engine.
Add memory for DB caches.
Add CPU for more intensive queries.

82. Scaling IO.
Provision throughput for your app.
Reserved. Available. Elastic.
Available on EBS, Amazon RDS and DynamoDB.

83. Provisioned throughput is consistent.
Consistent, predictable performance.
Relational databases with RDS.
NoSQL stores with DynamoDB.
Relational & NoSQL with EC2 and EBS.

84. Provisioned throughput with RDS.
12.5k IOPS on MySQL.
25k IOPS on Oracle.
10k IOPS on SQL Server.
Provision up to 30k for reduced latency.

85. Provisioned throughput and instance types.
Optimized for provisioned IO storage:
m1.large: 500 Mbps
m1.xlarge, m2.xlarge, m2.4xlarge: 1000 Mbps

86. Provisioned throughput and DynamoDB.
Consistent performance with
unlimited throughput and storage.
Single digital latency.

87. Build for a uniform workload.
Evenly distribute query patterns by key.
Use a wide range of key values.

88. 7
Content delivery
1
Horizontal scale
2
Instance selection
3
Caching
4
Read capacity
5 6

89. 7
Content delivery
1
Horizontal scale
2
Instance selection
3
Caching
4
Read capacity
5 6

90. Standard EBS volumes.
Moderate or bursty workloads.
100 IOPS, bursting to hundreds of IOPS.
Bursting is good for boot volumes.

91. Provisioned IOPS with EBS volumes.
Predictable, high performance for I/O-intensive workloads.
2000 IOPS per volume.
Stripe volumes for additional I/O.
Deliver within 10% of the performance, 99.9% of the time.

92. EBS-Optimized instances.
m1.large, m2.2xlarge, m3.xlarge: 500 Mbps
m1.xlarge, m2.4xlarge, m3.2xlarge, c1.xlarge: 1000 Mbps

93. High bandwidth networking
cg1.4xlarge, cc2.8xlarge, hi1.4xlarge, hs1.8xlarge and
cr1.8xlarge run on non-blocking, 10 gigabit networking.
Not EBS-Optimized, but can be used with
provisioned IOPS volumes.

94. High I/O instances
Designed for for high throughput database workloads.
2 x 1 TB SSDs
2 GB/s for reads
1.1 GB/s for writes

95. PVM instances
4 KB random reads: 120K IOPS
4 KB random writes: 10K–80K IOPS

96. HVM instances (including Windows)
4 KB random reads: 90K IOPS
4 KB random writes: 9K–75K IOPS

97. High Storage instances
High sequential I/O.
24 x 2 TB drives.
2.4 GB/s of 2 MiB sequential reads.
2.6 GB/s for sequential writes.

98. 7
Content delivery
1
Horizontal scale
2
Instance selection
3
Caching
4
Read capacity Block store
5 6

99. Back end Front end
%CPU
kbps
Initial connection SSL negotiation Time to first byte Content download

100. http://amzn.to/170S1VV http://amzn.to/Zod4PY

101. 14 rules for faster loading web sites
1. Make fewer HTTP requests. 8. Make JavaScript and CSS External.
2. Use a content delivery network. 9. Reduce DNS lookups.
3. Add an expires header. 10. Minify Javascript.
4. GZIP components. 11. Avoid redirects.
5. Put style sheets at the top. 12. Remove duplicate scripts.
6. Put scripts at the bottom. 13. Configure ETags.
7. Avoid CSS expressions. 14. Make AJAX cacheable.
http://stevesouders.com/hpws/rules.php

102. Front end optimization
7
Content delivery
1
Horizontal scale
2
Instance selection
3
Caching
4
Read capacity Block store
5 6

103. One more thing...

104. US East

105. Route 53
Sydney US East Dublin

106. Thank you!
matthew@amazon.com
@mza