AWS Summit 2011: Closing Keynote : The Story of Amazon.com's Move to the AWS Cloud
•
14 likes•1,940 views
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to AWS Summit 2011: Closing Keynote : The Story of Amazon.com's Move to the AWS Cloud
Similar to AWS Summit 2011: Closing Keynote : The Story of Amazon.com's Move to the AWS Cloud (10)
More from Amazon Web Services
More from Amazon Web Services (20)
Recently uploaded
Recently uploaded (20)
AWS Summit 2011: Closing Keynote : The Story of Amazon.com's Move to the AWS Cloud
- 1. amazon.com’s Journey to the Cloud Jon Jenkins jjenkin@amazon.com AWS Summit June 13, 2011
- 2. 1995 - 2010 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 3. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 4. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 5. + 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 6. First real data center 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 7. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 8. Distribution Center Isolation 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 9. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 10. Decouple Service Oriented Architecture Scale Horizontally Increase Speed of Execution Develop Iteratively Seek Simplicity 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 11. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 12. What could we do with just S3? 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 13. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 14. IMDB Widget Architecture Amazon Amazon Web Server IMDB Service Call Customer IMDB Service IMDB Database Render Process 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 15. The Problem • Release process is coupled • Runtime latency & scale requirements • Service integration issues The Solution • Use S3 as a service • Store raw HTML for the feature in S3 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 16. Before Amazon Amazon Web Server IMDB Service Call Customer IMDB Service IMDB Database Render Process After Amazon IMDB Amazon Web Server Generic S3 HTML S3 Customer Puller HTML Store 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 17. Results • Reduced page latency • IMDB doesn’t worry about scaling • Reduced web server CPU utilization • Improved availability through reduced dependencies • Simplified release model • AJAX readiness 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 18. What about a more complex case? 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 19. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 20. The Problem • The system has lots of moving parts • It must run in an external data center • It must scale up quickly • Development team is two people The Solution • Use as many AWS services as possible 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 21. CloudWatch SQS (latency, test case Alarm Rule results, status codes) Config Engines Store Agents Notification Notification Web Portal Scheduler S3 Configuration (primary/ EC2 secondary) EC2 Web Service SDB ... EC2 RDS 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 22. Results • Very few dev resources required • Launched without having to negotiate any new datacenter co-lo presence • True external performance metrics • We can test site features in development that have not yet launched • The system scales horizontally to large amounts of traffic 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 23. What about amazon.com web servers? 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 24. Typical Weekly Traffic to amazon.com Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 25. Typical Weekly Traffic to amazon.com Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 26. Typical Weekly Traffic to amazon.com 39% 61% Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 27. November Traffic for amazon.com 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 28. November Traffic for amazon.com 76% 24% 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 29. The Problem • Retail web site hardware is underutilized • Traffic spikes require heroic effort • Scaling is non-linear The Solution • Migrate the entire www.amazon.com web server fleet to AWS 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 30. amazon.com AWS Availability Zone 1 ... EC2 www 1 EC2 www N Load Balancer Customer ... Availability Zone N VPC ... Databases Services EC2 www 1 EC2 www N 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 31. November 10, 2010 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 32. Results • All traffic for www.amazon.com is now served from AWS • We can dynamically scale the fleet in increments as small as a single host • Traffic spikes can be handled with ease 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 33. What about a DB use case? 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 34. 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 35. Basic Order Storage Architecture Order Database Web Servers Ordering Service 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 36. Basic Order Storage Architecture Order Database Web Servers Ordering Service Scaling Pattern 1 Scaling Pattern 2 Order Order Database Database Web Servers Ordering Service Web Servers Ordering Service 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 37. The Problem • Cumulative data impacts scale • No database scaling pattern is ideal • Databases infrastructure is expensive The Solution • Create a tiered storage system with AWS 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 38. Order Database Web Servers Ordering Service S3 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 39. Results • 670 million (4TB) orders now stored in S3 • We are spending way less on DB hosts • Sets us up for migration to RDS / SDB 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 40. Lessons learned 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 41. Business Lessons • Less time spent on capacity planning • Fewer conversations with finance • More innovation • Happier developers • I get credit for AWS price reductions • Be sure to consider compliance issues • No more lease returns! 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 42. Technical Lessons • Start with simple applications • Iterate toward your desired end-state • Identify reusable components • Engage security early and treat them as partners • Migrate to the cloud in concert with your other architectural objectives • The cloud can’t cover up sloppy engineering 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
- 43. Q&A 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011