AWS 101 Event - 16 July 2013

1. AWS 101 Ianni Vamvadelis Manager, Solution Architecture

2. Agenda 10:00 - 10:45 Presentation – AWS 101 Introducing the concepts behind AWS such as utility computing and elasticity. 10:45 - 11:00 Coffee Break 11:00 - 11:45 Presentation and Demonstration Live demonstration and interactive walkthrough

3. What we are going to cover Keypairs Security groups EC2 instances Metadata service Autoscaling Amazon Machine Images S3 CloudFront Elastic Load balancer RDS Feel free to follow along on your laptops

4. background

5. Consumer Business Tens of millions of active customer accounts Eight countries: US, UK, Germany, Japan, France, Canada, China, Italy Seller Business Sell on Amazon websites Use Amazon technology for your own retail website Leverage Amazon’s massive fulfillment center network IT Infrastructure Business Cloud computing infrastructure for hosting web-scale solutions Hundreds of thousands of registered customers in over 190 countries

6. Deep experience in building and operating global web scale systems About Amazon Web Services ? …get into cloud computing? How did Amazon…

7. Over 10 years in the making Enablement of sellers on Amazon Internal need for scalable deployment environment Early forays proved developers were hungry for more

8. AWS Mission Enable businesses and developers to use web services* to build scalable, sophisticated applications. *What people now call “the cloud”

9. Not excess capacity!

10. Each day AWS adds the equivalent server capacity to power Amazon when it was a global, $5B enterprise

11. Each day AWS adds the equivalent server capacity to power Amazon when it was a global, $5B enterprise $5.2B retail business 7,800 employees A whole lot of servers 2003

12. Total Number of Objects Stored in Amazon S3 Objects in S3 2.9 Billion 14 Billion 40 Billion 102 Billion 262 Billion 762 Billion 1.7 Trillion 2 Trillion Q4 2006 Q4 2007 Q4 2008 Q4 2009 Q4 2010 Q4 2011 Q4 2012 Q2 2013 Over 1.1 Million requests per second

13. EMR Jobs 0 500,000 1,000,000 1,500,000 2,000,000 2,500,000 3,000,000 3,500,000 4,000,000 3.7 M clusters launched since May 2010

14. Utility computing

15. On demand Pay as you go Uniform Available Utility computing

16. Utility computing On demand Pay as you go Uniform Available

17. Utility computing

18. Utility computing Compute Storage Security Scaling Database Networking Monitoring Messaging Workflow DNS Load Balancing BackupCDN On demand Pay as you go Uniform Available

19. On a global footprint Region US-WEST (N. California) EU-WEST (Ireland) ASIA PAC (Tokyo) ASIA PAC (Singapore) US-WEST (Oregon) SOUTH AMERICA (Sao Paulo) US-EAST (Virginia) GOV CLOUD ASIA PAC (Sydney)

20. Availability Zone On a global footprint

21. Edge Locations Dallas(2) St.Louis Miami JacksonvilleLos Angeles (2) Palo Alto Seattle Ashburn(2) Newark New York (3) Dublin London(2) Amsterdam (2) Stockholm Frankfurt(2) Paris(2) Singapore(2) Hong Kong Tokyo (2) Sao Paulo South Bend San Jose Osaka Milan Sydney On a global footprint Madrid

22. At the end of a web service ec2-run-instances ami-b232d0db --instance-count 3 --availability-zone eu-west-1a --instance-type m1.small ec2-run-instances ami-b232d0db --instance-count 5 --availability-zone eu-west-1c --instance-type m1.medium

23. ec2-run-instances ami-b232d0db --instance-count 2 --availability-zone us-east-1b --instance-type m1.xlarge ec2-run-instances ami-b232d0db --instance-count 2 --availability-zone us-east-1d --instance-type m1.xlarge At the end of a web service

24. At the end of a web service elb-create-lb myLoadBalancer as-create-auto-scaling-group MyGroup --launch-configuration MyConfig --availability-zones eu-west-1c --min-size 2 --max-size 200 ec2-authorize default -p 80

25. and a rich Management Console

26. Elasticity

27. Traditional IT capacity Elastic capacity Capacity Time Your IT needs

28. On and Off Fast Growth Variable peaks Predictable peaks Elastic capacity

29. Elastic capacity On and Off Fast Growth Predictable peaksVariable peaks WASTE CUSTOMER DISSATISFACTION

30. Elastic cloud capacity Traditional IT capacity Your IT needs Time Capacity Elastic capacity

31. Elastic capacity Fast GrowthOn and Off Predictable peaksVariable peaks

32. 503 Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

33. 503 Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

34. From one instance…

35. …to thousands

36. and back…

37. Customers elasticity

38. Sunday Monday Tuesday Wednesday Thursday Friday Saturday Typical weekly traffic to Amazon.com

39. November traffic to Amazon.com November

40. November traffic to Amazon.com Provisioned capacity November

41. November traffic to Amazon.com 76% 24% Provisioned capacity November

42. November 10th 2010 Turned off last physical web server of Amazon.com

43. November 10th 2010 Turned off last physical web server of Amazon.com October 31st 2011 Turned off last web servers supporting European business

44. November traffic to Amazon.com November

45. Time: +00h <10 cores Elastic Capacity

46. Time: +24h >1500 cores Elastic Capacity

47. Time: +72h <10 cores Elastic Capacity

48. Time: +120h >600 cores Elastic Capacity

49. NumberofEC2Instances 4/12/2008 4/14/2008 4/15/2008 4/16/2008 4/18/2008 4/19/2008 4/20/20084/17/20084/13/2008 40 servers to 5000 in 3 days EC2 scaled to peak of 5000 instances “Techcrunched” Launch of Facebook modification Steady state of ~40 instances

50. The Toolbox

51. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Reference Model security

52. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Global infrastructure Regions An independent collection of AWS resources in a defined geography A solid foundation for meeting location-dependent privacy and compliance requirements

53. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Global infrastructure Availability Zones Designed as independent failure zones Physically separated within a typical metropolitan region

54. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Global infrastructure Edge Locations To deliver content to end users with lower latency A global network of edge locations Supports global DNS infrastructure (Route53) and Cloud Front CDN

55. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Networking Direct Connect Dedicated connection to AWS VPN Connection Secure internet connection to AWS Virtual Private Cloud Private, isolated section of the AWS Cloud Route 53 Highly available and scalable Domain Name Service

56. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Compute Vertical Scaling From $0.02/hr Elastic Compute Cloud (EC2) Basic unit of compute capacity Range of CPU, memory & local disk options 13 Instance types available, from micro to cluster compute Feature Details Flexible Run windows or linux distributions Scalable Wide range of instance types from micro to cluster compute Machine Images Configurations can be saved as machine images (AMIs) from which new instances can be created Full control Full root or administrator rights Secure Full firewall control via Security Groups Monitoring Publishes metrics to Cloud Watch Inexpensive On-demand, Reserved and Spot instance types VM Import/Export Import and export VM images to transfer configurations in and out of EC2

57. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Compute Auto-scaling Automatic provisioning of compute resources based upon demand, configuration or schedule Trigger auto- scaling policy Feature Details Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs Integrated to CloudWatch Use metrics gathered by CloudWatch to drive scaling Instance types Run auto scaling for on-demand instances and spot. Compatible with VPC as-create-auto-scaling-group MyGroup --launch-configuration MyConfig --availability-zones eu-west-1a --min-size 4 --max-size 200

58. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Compute Elastic Load Balancing Create highly scalable applications Distribute load across EC2 instances in multiple availability zones Feature Details Auto-scaling Automatically scales to handle request volume Available Load balance across instances in multiple availability zones Health checks Automatically checks health of instances and takes them in or out of service Session stickiness Route requests to the same instance Secure sockets layer Supports SSL offload from web and application servers with flexible cipher support Monitoring Publishes metrics to Cloud Watch

59. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Storage S3 - Durable storage, any object 99.999999999% durability of objects Unlimited storage of objects of any type Up to 5TB size per objectFeature Details Flexible object store Buckets act like drives, folder structures within Access control Granular control over object permissions Server-side encryption 256bit AES encryption of objects Multi-part uploads Improved throughput & control Object versioning Archive old objects and version new ones Object expiry Automatically remove old objects Access logging Full audit log of bucket/object actions Web content hosting Serve content as web site with built in page handling Notifications Receive notifications on key events Import/Export Physical device import/export service

60. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Storage Elastic Block Store High performance block storage device 1GB to 1TB in size Mount as drives to instances Feature Details High performance file system Mount EBS as drives and format as required Flexible size Volumes from 1GB to 1TB in size Secure Private to your instances Available Replicated within an Availability Zone Backups Volumes can be snapshotted for point in time restore Monitoring Detailed metrics captured via Cloud Watch

61. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Database Relational Database Service Database-as-a-Service No need to install or manage database instances Scalable and fault tolerant configurations Feature Details Platform support Create MySQL, SQL Server and Oracle RDBMS Preconfigured Get started instantly with sensible default settings Automated patching Keep your database platform up to date automatically Backups Automatic backups and point in time recovery and full DB backups Backups Volumes can be snapshotted for point in time restore Failover Automated failover to slave hosts in event of a failure Replication Easily create read-replicas of your data and seamlessly replicate data across availability zones

62. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Database Amazon Relational Database Service (Amazon RDS) databases stores forum threads, site content, and project configuration data. High availability Multi-AZ database deployment to handle live game metadata and user-generated content. Enterprise-grade fault tolerance for protecting customer data. By managing time-consuming database administration tasks, Amazon RDS allows SEGA to focus on business critical applications.

63. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Database DynamoDB Provisioned throughput NoSQL database Fast, predictable performance Fully distributed, fault tolerant architecture Feature Details Provisioned throughput Dial up or down provisioned read/write capacity Predictable performance Average single digit millisecond latencies from SSD backed infrastructure Strong consistency Be sure you are reading the most up to date values Fault tolerant Data replicated across availability zones Monitoring Integrated to Cloud Watch Secure Integrates with AWS Identity and Access Management (IAM) Elastic MapReduce Integrates with Elastic MapReduce for complex analytics on large datasets

64. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Application Services Amazon SQS Processing task/processing trig Processing results Amazon SQS Reliable, highly scalable, queue service for storing messages as they travel between instances Feature Details Reliable Messages stored redundantly across multiple availability zones Simple Simple APIs to send and receive messages Scalable Unlimited number of messages Secure Authentication of queues to ensure controlled access

65. Task A Task B (Auto-scaling) Task C 2 3 1 Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Application Services Feature Details Process state Maintain application state across complex workflows in a reliable and available manner Tracking Tracks executions and log process for audit purposes Consistency Ensures processing tasks are executed and duplicity of events does not occur Simple Simple Decider and Task programming model for rapid integration Simple Workflow Reliably coordinate processing steps across applications Integrate AWS and non-AWS resources Manage distributed state in complex systems

66. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Application Services Cloud Search Elastic search engine based upon Amazon A9 search engine Fully managed service with sophisticated feature set Scales automatically Document Server Results Search Server Feature Details Auto-scaling Automatically scales based upon request volumes and data volumes High performance In memory operation means consistently low latency for search results Sophisticated features Support for faceting, stemming, synonyms, stop words and custom rank expressions Low cost Elastic service, pay for what you use

67. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Application Services CloudFront World-wide content distribution network Easily distribute content to end users with low latency, high data transfer speeds, and no commitments. Feature Details Fast Multiple world-wide edge locations to serve content as close to your users as possible Integrated with other services Works seamlessly with S3 and EC2 origin servers Dynamic content Supports static and dynamic content from origin servers Streaming Supports rtmp from S3 and includes support for live streaming from Adobe FMS and Microsoft Media Server London Paris NY Served from S3 /images/* 3 Served from EC2 *.php 2 Single CNAME www.mysite.com 1

68. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Deployment & Admin Elastic Beanstalk One-click deployment from Eclipse, Visual Studio and Git Rapid deployment of applications All AWS resources automatically created Feature Details Platform support Containers for Java, .net and PHP Resource creation Creates load balancer, instances, autoscaling and monitoring automatically Monitoring & Logs Integrated with Cloud Watch and consolidates server logs Versioning Manage versions of applications and easily rollback deployments Notifications Receive alerts on key events Full resource access Access all underlying AWS resources as necessary

69. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Deployment & Admin OpsWorks DevOps focused managed application stacks Underlying Chef recipes allow for complete customisation Feature Details Platform support Chef recipes allows for community expansion for platform components such as Solr, NgniX etc Resource creation Customizable deployments, rollback, partial deployments, patch management, automatic instance scaling, and auto healing Layered Manage logical application layers and combine into stacks.

70. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Deployment & Admin Cloud Formation Automate creation of ‘stacks’ in a repeatable way Scripting framework for AWS resource creation Feature Details Platform support Support for AWS resources from EC2 to IAM Resource creation Creates AWS resources behind the scenes and reports on progress Declarative Specify stacks in JSON format and source control your environments Customizable Drive stack creation with paramaters

71. Compute Storage AWS Global Infrastructure Database App Services Deployment & Administration Networking Deployment & Admin Identity & Access Management Granular control of user rights with AWS Automated granting of EC2 service rights Software Developer Kits Comprehensive support of programming models for using AWS services

72. + others Simple Email Service Simple Notification Service ElastiCache Elastic MapReduce CloudWatch …

73. Introducing Amazon Redshift Data Warehousing the AWS Way Easily and rapidly analyze petabytes of data 1/10 the cost of traditional data warehouses Automated deployment & administration Compatible with popular BI tools

74. Orchestration service for data-driven workflows Create automated and scheduled data flows Pre-integrated with AWS data sources Easily connect with 3rd party & on-premises sources AWSData Pipeline Your Analytics

75. Amazon S3 Input Bucket Amazon S3 Input Bucket Amazon S3 Output Bucket Amazon S3 Output Bucket Amazon S3 Output Bucket Transcoding Pipeline 1 Transcoding Pipeline 2 Transcoding Pipeline 4 Kindle Fire HD720p Custom    Inside a Transcoding Pipeline Transcoding Job n Transcoding Job 2 Transcoding Job 1 Amazon Elastic Transcoder

76. Security & Compliance

77. Foundation Services Compute Storage Database Networking AWS Global Infrastructure Regions Availability Zones Edge Locations Amazon Shared responsibility

78. Foundation Services Compute Storage Database Networking AWS Global Infrastructure Regions Availability Zones Edge Locations Client-side Data Encryption & Data Integrity Authentication Server-side Encryption (File System and/or Data) Network Traffic Protection (Encryption/Integrity/Identity) Platform, Applications, Identity & Access Management Operating System, Network & Firewall Configuration Customer Data Amazon Shared responsibility You

79. Certifications SOC 1 Type 2 (formerly SAS-70) ISO 27001 PCI DSS for EC2, S3, EBS, VPC, RDS, ELB, IAM FISMA Moderate Compliant Controls HIPAA & ITAR Compliant Architecture Physical Security Datacenters in nondescript facilities Physical access strictly controlled Must pass two-factor authentication at least twice for floor access Physical access logged and audited HW, SW, Network Systematic change management Phased updates deployment Safe storage decommission Automated monitoring and self-audit Advanced network protection Security standards http://aws.amazon.com/security

80. So what are we going to do today?

81. Availability Zone Region Instance

82. Availability Zone Region Instance S3

83. Availability Zone Region Instance S3 Cloud Front

84. Availability Zone Region Instance S3 Cloud Front RDS

85. Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 RDS

86. Availability Zone Region Instance Instance Elastic Load Balancer Cloud Front S3 Auto scaling Group RDS

87. Enjoy some coffee / tea, come back at 11:00, and we’ll build it

88. Ask questions (it will fill time when we wait for things to launch) There will be a recording of this demo on YouTube, so don’t worry if you miss anything

93. EC2 launch an instance

94. bootstrapping passing data to an instance

95. Instance request User data

96. Instance request User data Meta-data service

97. Instance request User data Instance Meta-data service

98. Script executed on launch: <script> ipconfig /all > c:ipconfig.txt netstat > c:netstat.txt </script>

99. Script executed on launch: #!/bin/sh yum -y install httpd php php-mysql chkconfig httpd on /etc/init.d/httpd start

100. key pairs secure access

101. Public Key Inserted by Amazon into each EC2 instance that you launch Private Key Downloaded and stored by you EC2 Instance Comms secured with private key

102. security groups instance firewalling

103. Security Group instance Port 80 (HTTP) Port 22 (SSH) Name Description Protocol Port range IP Address, range, or another security group

104. index.php Reads instance meta-data

105. Some php code that gets the data <?php // get the instance id $url = "http://169.254.169.254/latest/meta-data/instance-id"; $instance_id = file_get_contents($url); // get the AZ where the instance is running $url = "http://169.254.169.254/latest/meta-data/placement/availability-zone"; $zone = file_get_contents($url); // get the security group it is in $url = "http://169.254.169.254/latest/meta-data/security-groups"; $group = file_get_contents($url); // get the public DNS name $url = "http://169.254.169.254/latest/meta-data/public-hostname"; $hostname = file_get_contents($url); ?>

106. And diplays it Instance ID: <?php echo $instance_id; ?> Availability Zone: <?php echo $zone; ?> Security Group: <?php echo $group; ?>

107. ELB elastic load balancer

108. Availability Zone Availability Zone Region Availability Zone Instance Instance Instance Instance Instance Instance Elastic Load Balancer

110. auto-scaling elastic server pool

112. Describes what Auto Scaling will create when adding Instances AMI Instance Type Security Group Instance Key Pair Only one active launch configuration at a time Auto Scaling will terminate instances with old launch configuration first rolling update Auto Scaling managed grouping of EC2 instances Automatic health check to maintain pool size Automatically scale the number of instances by policy – Min, Max, Desired Automatic Integration with ELB Automatic distribution & balancing across AZs Parameters for performing an Auto Scaling action Scale Up/Down and by how much ChangeInCapacity (+/- #) ExactCapacity (#) ChangeInPercent (+/- %) Cool Down (seconds) Policy can be triggered by CloudWatch events Launch Configuration Auto-Scaling Group Auto-Scaling Policy

113. as-create-launch-config --image-id ami-50348c51 --instance-type t1.micro --key apac-tok --group 'Web Servers’ --launch-config 101-launch-config Create a launch configuration:

114. as-create-auto-scaling-group 101-as-group --availability-zones ap-northeast-1a --launch-configuration 101-launch-config --load-balancers myELB --max-size 5 --min-size 1 Create an auto-scaling group:

115. as-put-scaling-policy 101ScaleUpPolicy --auto-scaling-group 101-as-group --adjustment=1 --type ChangeInCapacity --cooldown 300 Create an auto-scaling policy (scale up):

116. as-put-scaling-policy 101ScaleDownPolicy --auto-scaling-group 101-as-group "--adjustment=-1" --type ChangeInCapacity --cooldown 300 Create an auto-scaling policy (scale down):

118. To do this you will need: An AWS Account Auto-scaling tools installed Find out more: aws.amazon.com

119. aws.amazon.com

AWS 101 Event - 16 July 2013

Esté a ler uma amostra.

Confira estes a seguir

AWS 101 Event - 16 July 2013

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a AWS 101 Event - 16 July 2013 (20)

Mais de Amazon Web Services (20)

Mais recentes (20)