RightScale Webinar: December 14, 2010 – In this webinar, we show you how to use the cloud to build websites that scale to meet demand.

1. How to BuildScalable Web Applicationsin the CloudReference Architectures and Best PracticesDecember 14, 2010

5. Comparison of traditional and cloud resource models

6. Reference architecture for a scalable web application

7. Best practices for each tier of the reference architecture

8. Summary & conclusionsPlease use the questions window to ask questions any time!

9. Scalable Web Application What? An application built on an architecture that can adapt to changing conditions

10. Scalable Web Application What? An application layered on an architecture that can adapt to changing conditions Why? Traffic and load patterns are unpredictable Viral or flash-mob events can result in very dynamic conditions Availability and Reliability Application must be distributed to increase likelihood of end-user accessibility Overprovision Under-utilized resources == wasted $$$ Underprovision Missed opportunities – users unable to access your site/product Don’t be a victim of your own success

11. Pre-Cloud Provisioning

12. Cloud-enabled Provisioning

13. Reference Architecture

14. Load Balancing Tier

15. Load Balancing HAProxy + Apache Can handle SSL termination on the load balancer Connection statistics available via socket connection and status web page Each instance can handle a specific amount of traffic with no ramp-up time Each instance can only handle a specific amount of traffic Addition of load balancers is possible, but requires DNS modifications

16. Load Balancing HAProxy + Apache Can handle SSL termination on the load balancer Connection statistics available via socket connection and status web page Each instance can handle a specific amount of traffic with no ramp-up time Each instance can only handle a specific amount of traffic Addition of load balancers is possible, but requires DNS modifications Elastic Load Balancer (ELB) SSL termination is now supported Can scale to handle large amounts of traffic, but can be slow to ramp-up Options do exist for “pre-warming” the ELB Only need one – it will scale to accommodate traffic load Essentially a load balancing appliance No visibility into inner-workings and/or connection rates, statistics, failures, etc. (RightScale has a technical white paper on load balancing solutions that is available at www.RightScale.com/whitepapers)

17. Load Balancing Load Balancer + Application server Possible, and good for test and dev Not a best practice for a production environment Traffic spikes can cause instance to perform both load balancing and application functions…poorly

18. Load Balancing Load Balancer + Application server Possible, and good for test and dev Not a best practice for a production environment Traffic spikes can cause instance to perform both load balancing and application functions…poorly Recommendation: Minimum of two load balancers Each load balancer should be in a different availability zone (AZ) to increase reliability and availability RightScale testing has shown that m1.large is a good choice for load balancers Due to 100K-120K packet-per-second limit, larger instances do not provide much gain in throughput Roughly 5K responses/second can be handled by m1.large With the 5K threshold in mind, select the number of load balancers required to handle your peak traffic

19. Application Server Tier Puts the “scalable” in a scalable application True autoscaling a must in any dynamic/unpredictable environment

20. Application Server Tier Autoscaling Fully automated server launch based on autoscaling triggers No manual intervention (can be challenging in certain environments, i.e. Windows) Download and install application code from common repository to ensure identical configuration of all servers in the tier

21. Application Server Tier Autoscaling Fully automated server launch based on autoscaling triggers No manual intervention (can be challenging in certain environments, i.e. Windows) Download and install application code from common repository to ensure identical configuration of all servers in the tier Triggers Common CPU idle Free memory System load Custom Web server connections Application-specific metrics

22. Application Server Tier When to scale? Conservatively. Both up and down

23. Application Server Tier When to scale? Conservatively. Both up and down Up Allow adequate lead time for new servers to become operational Before system is negatively impacted Look for trends in activity and react early Worst that can happen: Charged for an extra instance hour

24. Application Server Tier When to scale? Conservatively. Both up and down Up Allow adequate lead time for new servers to become operational Before system is negatively impacted Look for trends in activity and react early Worst that can happen: Charged for an extra instance hour Down When system has been underutilized for a consistent, consecutive period of time Scale down fewer servers than in a scale-up event Again, only downside is an extra hour of instance charge Better safe than sorry

25. Application Server Tier Array considerations

26. Application Server Tier Array considerations Weight the array across all availability zones (not regions) Increases reliability of application NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte charge Traffic between regions is charged at public Internet rates

27. Application Server Tier Array considerations Weight the array across all availability zones (not regions) Increases reliability of application NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte charge Traffic between regions is charged at public Internet rates Set minimums and maximums appropriately Minimum can assist in cost savings in times of low usage Maximum can limit overall cost exposure

28. Application Server Tier Array considerations Weight the array across all availability zones (not regions) Increases reliability of application NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte charge Traffic between regions is charged at public Internet rates Set minimums and maximums appropriately Minimum can assist in cost savings in times of low usage Maximum can limit overall cost exposure Instance size m1.large is typically a good choice for array-based servers in a production environment m1.smalls (and even micro instances) can be used in test and development environments Every application is different, so run load tests and benchmarks to find the optimal solution for your environment

29. Application Server Tier Array considerations Weight the array across all availability zones (not regions) Increases reliability of application NOTE: Traffic within an AZ on private IPs is free. Traffic between AZs incurs a per-gigabyte charge Traffic between regions is charged at public Internet rates Set minimums and maximums appropriately Minimum can assist in cost savings in times of low usage Maximum can limit overall cost exposure Instance size m1.large is typically a good choice for array-based servers in a production environment m1.smalls (and even micro instances) can be used in test and development environments Every application is different, so run load tests and benchmarks to find the optimal solution for your environment Code Deployment Updated code can be pushed to all servers in an array via a single click of a button

30. Caching Tier Caching can dramatically decrease the load on the database Particularly in read-intensive applications Costs of caching Application complexity/modification Additional instance hours to support the cache

31. Caching Tier Best practice is to have a separate, dedicated caching tier Caching can be implemented on each application server Prevents the use of a distributed cache Local cache should only be used by the co-resident application server Application complexities Database performance degradation

32. Caching Tier Best practice is to have a separate, dedicated caching tier Caching can be implemented on each application server Prevents the use of a distributed cache Local cache should only be used by the co-resident application server Application complexities Database performance degradation Instance Size and Count Determine memory caching footprint Select instance size and count that spreads the load over several servers Prevents loss of entire cache if a single instance fails Distribute caching servers across AZs for reliability Overprovision if possible Provide capacity for system to grow to fully utilize cache (budget permitting)

33. Caching Tier Best practice is to have a separate, dedicated caching tier Caching can be implemented on each application server Prevents the use of a distributed cache Local cache should only be used by the co-resident application server Application complexities Database performance degradation Instance Size and Count Determine memory caching footprint Select instance size and count that spreads the load over several servers Prevents loss of entire cache if a single instance fails Distribute caching servers across AZs for reliability Overprovision if possible Provide capacity for system to grow to fully utilize cache (budget permitting) Manually scaling caching servers is possible but non-trivial Involves application and database performance degradation Time To Lives (TTLs) Always set to expire

34. Caching Tier Write-intensive applications Don’t see as large a performance gain as read-intensive apps

35. Caching Tier Write-intensive applications Don’t see as large a performance gain as read-intensive apps Third-party providers Vendor solutions exist that allow dynamic memcached scaling

36. Database Tier Numerous database architecture options exist No “one size fits all” solution, so testing and benchmarking are critical to determine proper configuration

37. Database Tier Masters and Slave(s) Multiple Slaves if budget permits Distribute Master and Slave(s) across AZs Always use same instance size for Master and Slaves

38. Database Tier Masters and Slave(s) Multiple Slaves if budget permits Distribute Master and Slave(s) across AZs Always use same instance size for Master and Slaves Data Storage EBS volumes for data store Never use ephemeral storage for persistent data Back up Master and Slaves frequently Upload snapshots to S3 or some other persistent, redundant storage

39. Database Tier Masters and Slave(s) Multiple Slaves if budget permits Distribute Master and Slave(s) across AZs Always use same instance size for Master and Slaves Data Storage EBS volumes for data store Never use ephemeral storage for persistent data Back up Master and Slaves frequently Upload snapshots to S3 or some other persistent, redundant storage Instance Size Varies greatly based on the nature of the application and site traffic Load testing and benchmarking can assist in identifying a reasonable initial size

40. Database Tier Relational Database Service (RDS) Database Appliance No access to instance (no visibility into CPU utilization, memory usage, slow-query logs, etc.) Requires scheduled downtime Announcement of multi-AZ functionality in May 2010 allows 24/7 operation Read replicas announced in October 2010

41. Database Scaling

42. Database Scaling Vertical “Grow” or “shrink” a database server from one instance size to another

43. Database Scaling Vertical “Grow” or “shrink” a database server from one instance size to another Horizontal Add additional servers to spread the database load

44. Database Vertical/Horizontal Scaling

45. Horizontal Database Scaling Addition of read Slaves Effective for read-intensive applications Only writes need to access the master Replication lag to slaves must be considered

46. Horizontal Database Scaling Addition of read Slaves Effective for read-intensive applications Only writes need to access the master Replication lag to slaves must be considered Effective mechanism is to use MySQL Proxy

47. Horizontal Database Scaling Sharding Concept is to partition the database into distinct, non-overlapping pieces “Horizontal slicing” of the database tables into groups of rows Forethought required in setting up shards since cross-shard joins are resource intensive

48. Horizontal Database Scaling Before Sharding

49. Horizontal Database Scaling After Sharding

50. Horizontal Database Scaling Master-Master Two (or more) Master DBs Any Master can modify any database object Replication lag can result in database inconsistencies Poorly-designed applications can cause data object collisions and leave databases in indeterminate state Not a recommended best practice, nor supported by RightScale

51. Horizontal Database Scaling NoSQL solutions Many options exist – SimpleDB, Cassandra, Membase, CouchDB, MongoDB, Riak, etc. Basically a Key/Value store No complex operations between data objects (no relational operations) Multiple nodes can be used to implement a distributed data store Coordinated backup and recovery can be challenging Some RightScale customers are beginning to use some NoSQL solutions in specific use cases.

52. So What’s Best?

53. So What’s Best? As is typical in many technology discussions…

54. So What’s Best? As is typical in many technology discussions… “It depends”

55. So What’s Best? As is typical in many technology discussions… Many scalable environments share some common underlying architecture concepts “It depends”

56. So What’s Best? As is typical in many technology discussions… Many scalable environments share some common underlying architecture concepts Every application is different. As such, there is no “one size fits all” “It depends”

57. So What’s Best? As is typical in many technology discussions… Many scalable environments share some common underlying architecture concepts Every application is different. As such, there is no “one size fits all” But… “It depends”

58. Mix-n-Match Pre-Built Components

59. Q&A / Getting Started Have a project, but need some help? Contact us: sales@rightscale.com or (866) 720-0208 Ready to get started? Sign up for our Free Edition:www.RightScale.com/Free Call us for a VIP trial of our paid editions Need to learn more? Scalable Web Apps white paper – Coming Soon! TCO calculator:www.RightScale.com/tco-calculator Webinar archive: www.RightScale.com/webinars White papers: www.RightScale.com/whitepapers

60. Thank You!