O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Tech Talks On Site- Edição de Maio- AutoScaling

90 visualizações

Publicada em

Apresentação do evento Tech Talks On Site no mês de maio de 2019.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Tech Talks On Site- Edição de Maio- AutoScaling

  1. 1. 5/21/19 1 © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Mv – Marcus Vinicius Ferreira / Claick Oliveira Solution Architect Team, Public Sector, Education Maio/2019 AWS: AutoScaling Compute, AutoScaling, DevOps Mv – Marcus Vinicius Ferreira mvferr@amazon.com Solutions Architect BR, Public Sector, Education Previous: Oracle, Sun, Abril, Dinda, NubankMv
  2. 2. 5/21/19 2 Claick Oliveira claicko@amazon.com Solutions Architect BR, Public Sector, Education Claick AWS Agenda AWS é Escala AWS Compute: EC2 AWS AutoScaling AWS Automation: DevOps AWS Casos de Uso
  3. 3. 5/21/19 3 AWS is Architected for Government Security Requirements Certifications and accreditations for workloads that matter – Compliant Solutions AWS CloudTrail and AWS Config – Call logging and configuration management for governance and compliance • Log, review, alarm on all user actions • Browse-and-query database of current and previous state of cloud resources MTCS https://aws.amazon.com/compliance/ What Is (True) Cloud Computing? The on-demand delivery of IT resources over public or private networks with zero up-front costs, no long-term contracts, and pay-as-you-go pricing 6
  4. 4. 5/21/19 4 Service Breadth & Depth TECHNICAL & BUSINESS SUPPORT Account Management Support Professional Services Training & Certification Security & Pricing Reports Partner Ecosystem Solutions Architects ENTERPRISE APPS Virtual Desktops Sharing & Collaboration Corporate Email Backup Regions Availability Zones Points of Presence INFRASTRUCTURE Compute Storage DatabasesCDN Networking CORE SERVICES HYBRID ARCHITECTURE Data Backups Integrated App Deployments Direct Connect Identity Federation Integrated Resource Management Integrated Networking Access ControlIdentity Key mgmt & Storage Monitoring & Logs SECURITY & COMPLIANCE Auditing Configuration, Compliance Firewalls Assessment, reporting MARKETPLACE Business Apps Business Intelligence Databases DevOps Tools NetworkingSecurity Storage IoT Rules Engine Device Shadows Device SDKs Registry Device Gateway DEV & OPSMOBILE SERVICESAPP SERVICESANALYTICS Data Warehouse Hadoop/Spark Data Collection Machine Learning Elastic Search Queuing & Notifications Workflow Search Email Transcoding One-click Deployment Identity Sync Single Integrated Console Push Notifications DevOps Application Lifecycle Management Containers Triggers Resource Templates API Gateway Data Analysis BI Mobile Analytics AWS é Escala
  5. 5. 5/21/19 5 AWS Global Infrastructure 18 Regions – 54 Availability Zones – 114 Edge Locations Region & Number of Availability Zones AWS GovCloud (2) EU Ireland (3) US West Frankfurt (2) Oregon (3) London (2) Northern California (3) Asia Pacific US East Singapore (2) N. Virginia (5), Ohio (3) Sydney (2), Tokyo (3), Seoul (2), Mumbai (2) Canada Central (2) China Beijing (2) South America São Paulo (3) Announced Regions Paris, Ningxia Region New Region Coming Soon Edge Location Region & Number of Availability Zones# The Global Infrastructure
  6. 6. 5/21/19 6 REGION ~ 2ms latency 3 AZa AZc AZb Redundant Tier-1 Internet & Inter-Region Connectivity TCTC DC DC DC DC DCDC DC DC DC Availability Zone 1a Availability Zone 1b Internet 10.0.0.5 10.0.0.6 10.0.3.17 10.0.3.5 10.0.1.5 10.0.1.25 10.0.1.8 10.0.1.6 VPC Subnet VPC Subnet VPC Subnet Virtual Private Gateway Customer Gateway VPN Connection Internet Gateway Customer Data Center Virtual Private Cloud
  7. 7. 5/21/19 7 Deploy however you like Your Datacenter Amazon Web Services Fully Featured Compute Resource & Deployment Management Common Controls for Security & Access Integrated Networking Data Integration & Life Cycle Management Flexible hybrid options Comcast’s IT strategy focuses on combining its own data centers and AWS as the cornerstone of its next-generation TV service, X1. This has allowed them to rapidly scale interactive, on-demand content to millions of viewers. AWS Compute: EC2
  8. 8. 5/21/19 8 G2 GPU enabled M5 General purpose Memory optimized R4 Dense-storage & High-I/O optimized C5 Compute optimized C4M4 D2 I2 Compute: EC2 Instance Families T3 Burstable performance X1P3 T2 Instance generation c5.xlarge Instance family Instance size
  9. 9. 5/21/19 9 Amazon EC2 Instances 256 128 64 32 16 8 4 2 1 1 2 4 8 16 32 64 128 Amazon EC2 Compute Units Memory(GB) High CPU High Mem ory Cluster Com pute and High I/O M icro General Purpose Cluster High Mem ory and High Storage Instance sizing c5.18xlarge 2 x c5.9xlarge ≈ 4 x c5.4xlarge ≈ 8 x c5.2xlarge ≈
  10. 10. 5/21/19 10 Machine Power Cluster compute instances Implement HVM process execution Intel® Xeon® processors 10 Gigabit Ethernet –c3 has Enhanced networking, SR-IOV cc2.8xlarge 32 vCPUs 2.6 GHz Intel Xeon E5-2670 Sandy Bridge 60.5 GB RAM 2 x 320 GB Local SSD c3.8xlarge 32 vCPUs 2.8 GHz Intel Xeon E5-2680v2 Ivy Bridge 60GB RAM 2 x 320 GB Local SSD AWS Auto Scaling
  11. 11. 5/21/19 11 Typical Weekly Traffic at Amazon.com Sunday Monday Tuesday Wednesday Thursday Friday Saturday Provisioned capacity November Traffic to Amazon.com Provisioned capacity November 76% 24% Challenge is to efficiently ‘guess’ the unknown quantity of how much compute capacity you need
  12. 12. 5/21/19 12 The Economics of the Cloud are Compelling Infrastructure cost $ Time The Economics of the Cloud are Compelling Infrastructure cost $ Time Predicted demand Key:
  13. 13. 5/21/19 13 The Economics of the Cloud are Compelling Infrastructure cost $ Time Large capital expenditure Predicted demand Traditional hardware Key: The Economics of the Cloud are Compelling Infrastructure cost $ Time Large capital expenditure Predicted demand Traditional hardware Actual demand Key:
  14. 14. 5/21/19 14 The Economics of the Cloud are Compelling Infrastructure cost $ Time Large capital expenditure Opportunity cost Predicted demand Traditional hardware Actual demand Key: The Economics of the Cloud are Compelling Lost opportunity Infrastructure cost $ Time Large capital expenditure Opportunity cost Predicted demand Traditional hardware Actual demand Key:
  15. 15. 5/21/19 15 The Economics of the Cloud are Compelling Lost opportunity Infrastructure cost $ Time Large capital expenditure Opportunity cost Predicted demand Traditional hardware Actual demand Automated virtualization Key: instance instanceinstance instance Auto Scaling group Minimum = 2 Maximum = 10 Desired # of instances = 4 Availability Zone bAvailability Zone a Elastic Load Balancing Elastic Load Balancing, CloudWatch, and Auto Scaling CloudWatch
  16. 16. 5/21/19 16 instance instanceinstance instance Auto Scaling group Minimum = 2 Maximum = 10 Desired # of instances = 4 Availability Zone bAvailability Zone a Elastic Load Balancing Elastic Load Balancing, CloudWatch, and Auto Scaling CloudWatch instance instanceinstance instance Auto Scaling group Minimum = 2 Maximum = 10 Desired # of instances = 4 Availability Zone bAvailability Zone a Elastic Load Balancing Elastic Load Balancing, CloudWatch, and Auto Scaling CloudWatch
  17. 17. 5/21/19 17 instance instanceinstance instance Auto Scaling group Minimum = 2 Maximum = 10 Desired # of instances = 6 instanceinstance Availability Zone bAvailability Zone a Elastic Load Balancing CloudWatch Elastic Load Balancing, CloudWatch, and Auto Scaling instance instanceinstance instance Auto Scaling group Minimum = 2 Maximum = 10 Desired # of instances = 6 instanceinstance Availability Zone bAvailability Zone a Elastic Load Balancing CloudWatch Unhealthy Instances Get Replaced…
  18. 18. 5/21/19 18 Unhealthy Instances Get Replaced… instance instanceinstance instance Auto Scaling group Minimum = 2 Maximum = 10 Desired # of instances = 6 instanceinstance Availability Zone bAvailability Zone a Elastic Load Balancing CloudWatch …In a Different AZ if Necessary instanceinstance instanceinstance Auto Scaling group Minimum = 2 Maximum = 10 Desired # of instances = 6 instance Availability Zone bAvailability Zone a instance Elastic Load Balancing CloudWatch
  19. 19. 5/21/19 19 Capacity matching Elastic Cloud-Based Resources Actual demand Resources scaled to demand Waste Customer Dissatisfaction Actual Demand Predicted Demand Rigid On-Premises Resources AWS Storage: EBS e S3 172.31.0.0/16 sa-east-1a sa-east-1b sa-east-1c
  20. 20. 5/21/19 20 Multi-AZ Architecture User Amazon Route 53 Internet Gateway Public Subnet Private Subnet Public Subnet Private Subnet Private Subnet Private Subnet Private Subnet BI / OLAP Public load balancer Private load balancer PROD / OLTP AWS AutoScaling: Components
  21. 21. 5/21/19 21 Elastic Load Balancing, CloudWatch, and Auto Scaling Latency CPU Utilization CloudWatchAuto Scaling Elastic Load Balancing Auto Scaling group Execute Lauch Configuration How Does Auto Scaling Work? Launch Configuration 1 Auto Scaling Group Auto Scaling Policy Scheduled Action 2 3 Launch configuration defines: • Name • AMI • Instance type • User data • Security groups • IAM role • Etc. Auto Scaling group defines: • Name • Launch configuration name • Min & Max • AZ or subnet • Load balancer • Desired capacity • Etc. Specifies when to dynamically increase or decrease Amazon EC2 instances based on CloudWatch alarms Tells Auto Scaling to perform a scaling action at a certain time in the future (minimum, maximum, and desired size for the ASG) EC2AMI Auto Scaling group Load balancer Auto Scaling group ? ? 1..N 1..20 What Where When
  22. 22. 5/21/19 22 How Do You Decide on Minimum Capacity Size? Auto Scaling group Availability Zone 1 Availability Zone 2 Auto Scaling group defines: Ø Desired capacity Ø Minimum capacity Ø Maximum capacity Do you have to specify desired capacity? What would be a good minimum capacity to set it to? What would be a good maximum capacity to set it to? ? Auto Scaling group Availability Zone 1 What about HA? Minimum = 2 instances (# of AZs) Desired capacity = 2 instances (Min.) 0 or 1? Maximum Capacity Size and Auto Scaling Scenario: Auto Scaling Group: Ø Minimum = 2 Ø Maximum = 12 Auto Scaling Policy: Ø When CPU utilization is greater than 60% Ø Add 100% of group = double the capacity Availability Zone 2Availability Zone 1 Auto Scaling group CPU utilization triggers the alarm: capacity is doubled until CPU utilization drops below 60% or max capacity is reached.
  23. 23. 5/21/19 23 AWS Building AMIs AMIs and Boot Times Remember the AMI balancing act! Test various configurations to find what best meets your baseline performance. OS-Only AMI • More dynamic • Slower boots Full AMI Partially Configured AMIs • Less dynamic • Faster boots Balance between ease of new deployments and boot load times
  24. 24. 5/21/19 24 AMI Creation Models Inventory of AMIs Golden AMI – Fetch Binaries on Boot JeOS AMI and Library of Recipes (Install Scripts) Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Linux JEE Your Code Log4J Spring Hibernate Struts Tomcat Apache Amazon EC2 Li n u x JE E Yo u r C o de Lo g 4 JSp r i n g Hi b e r n at e St r u t s To m c a t Ap a c h e Li n u x JE E Yo u r C o de Lo g 4 JSp r i n g Hi b e r n at e St r u t s To m c a t Ap a c h e Li n u x JE E Yo u r C o de Lo g 4 JSp r i n g Hi b e r n at e St r u t s To m c a t Ap a c h e Li n u x JE E Yo u r C o de Lo g 4 JSp r i n g Hi b e r n at e St r u t s To m c a t Ap a c h e Amazon EC2 Amazon EC2 Your Code Amazon S3 Log4J Spring Struts Linux JEE Hibernate Tomcat Apache Linux JEE Your Code Amazon S3 Hibernate Tomcat Log4J Spring Struts Apache Li n u x JE E Hi b e r n at e To m c a t Ap a c h e Li n u x JE E Hi b e r n at e To m c a t Ap a c h e Li n u x JE E Hi b e r n at e To m c a t Ap a c h e Linux JEE Linux JEE Chef/Puppet Chef/Puppet Scripts Java AMI Java App Stack Java AMI JeOS AMI Fetch on boot Fetch on boot Fetch on boot Minimal provisioning Partial provisioning on boot Full provisioning on boot Packer.io https://www.packer.io/downloads.html
  25. 25. 5/21/19 25 Ready. AMI. Fire! Linux AMI EC2: build machine. • Size: Medium • Run: repo update -y • Add: pkg: apache • Add: pkg: php • Add: pkg: mod_php • Add: pkg: memcache-client • Add: git checkout: my-app-release-1.2 • Add: wget: app/config.php • Add: wget: conf.d/my-app.conf Customer AMI • Name: my-app-1.2 Your LAN Segments Dev QA Prod Packer and command-line tools. AMI Approach Use Case: Netflix Uses a "tiered AMI" system with layered prerequisites. Foundation AMI (monitor agent, etc) Base AMI (Java) Application AMI (release 1.1) AMI provided by AWS Basic tools and system updates Core software and performance optimizations App-specific AMI generated by Jenkins CI platform AWS Linux AMI (Public AMI) Base AMI (Ruby Base AMI (Python) Application AMI (release 1.2) Application AMI (release x.x) Application AMI (release y.y)
  26. 26. 5/21/19 26 Packaging/baking AMIs #1 reason to bake is to decrease your boot time Ø Software packages that require painful/long setup Ø Standard software that must be there at startup Ø Any configuration items that cannot be remotely sourced or automated Strike a balance between those things that change often and those that don’t AWS provides easy interfaces to create the AMI or import the AMI Third-party tooling can be helpful • Packer (includes Linux and Windows) https://packer.io/ AMI Instances Tip: Starting from an existing Amazon-provided image is recommended. Once done customizing, you should stop the instance and capture the AMI. AWS Automation: DevOps
  27. 27. 5/21/19 27 Infrastructure Management Cloudformation
  28. 28. 5/21/19 28 DevOps: What is AWS CloudFormation? Declarative programming language for deploying AWS resources. Uses templates and stacks to provision resources. Create, update, and delete a set of resources as a single unit (stack). Create/delete AWS CloudFormation Create/delete AWS resources Template Stack - Basic definition of resources to create - JSON text file - Collection of AWS resources Example Environment Templates Dev Apps Stack Dev Base Stack Test Apps Stack Test Base Stack Private Subnet App tier Private Subnet DB tier Master Oracle Public Subnet Private Subnet Web tier Private Subnet App tier Private Subnet DB tier NAT Master Oracle AMIs Amazon EBS snapshots Internet Gateway Internet Gateway Development Account Production Account Private Subnet Web tier NAT Public Subnet
  29. 29. 5/21/19 29 Cloudformation to the RESCUE! AWS VPC Your LAN Segments AMI for Python AMI for Perl AMI for Java Remember: DO NOT share your machines! Integration with the cloud Your Data Center Project A Dev
  30. 30. 5/21/19 30 Dedicated Infrastructures Your Data Center AWS VPC Physical Cluster C++/Fortran Bio Informatics Perl Engineer Python Physics Java … and use dedicated clusters for specific software solutions Many Environments Development QA 1 QA 2
  31. 31. 5/21/19 31 Red-Black Deployment: Instant Cutover Web Server Fleet (Amazon EC2) ….. Load Balancing (Elastic Load Balancing) v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 Persistent Layer (Databases and S3) Red-Black Deployment: Pre-Cutover Web Server Fleet (Amazon EC2) ….. Persistent Layer (Databases and S3) Load Balancing (Elastic Load Balancing) v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1
  32. 32. 5/21/19 32 Red-Black Deployment: Cutover to New System Web Server Fleet (Amazon EC2) ….. Load Balancing (Elastic Load Balancing) v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 v1.1 v1.1 v1.1 v1.1 v1.1 v1.1 Persistent Layer (Databases and S3) Red-Black Deployment: Cutover to New System Web Server Fleet (Amazon EC2) Load Balancing (Elastic Load Balancing) v1.2 v1.2 v1.2 v1.2 v1.2 v1.2 Persistent Layer (Databases and S3)
  33. 33. 5/21/19 33 Embracing Failure: Fault Injection Build a strong test harness to force out-of-spec failures to surface. • Refuses all connections. • Reads requests at 1 byte/second. • Accepts request, and sends responses at 1 byte/second rate. • …etc. Inject failures regularly into your systems under controlled circumstances, using third-party tools such as Netflix Simian Army which includes Chaos Monkey, Chaos Gorilla, etc. Components: • EC2 Linux Auto-Scaling • RDS MySQL Multi-AZ • Elastic Load Balancer • S3 Bucket AWS: WordPress Reference Architecture
  34. 34. 5/21/19 34 AWS Elastic Beanstalk é a ferramenta orquestrador que executa um deploy a partir do Git, numa infra-estrutura em Auto-Scaling. AWS: Git, Elastic Beanstalk, Architecture Diferentes Sites terão: • Um repositório Git específico • Uma infra-estrutura Auto-Scaling dedicada • Uma rotina de deploy independente Múltiplos Sites: Git, Elastic Beanstalk, Deploy WebSite 1 WebSite 2 WebSite 3 WebSite 1 WebSite 2 WebSite 3
  35. 35. 5/21/19 35 AWS Beanstalk and Wordpress https://aws.amazon.com/getting-started/projects/build-wordpress-website/ Questions? Mv – mvferr@amazon.com
  36. 36. 5/21/19 36 Website! Obrigado! Mv – mvferr@amazon.com

×