Continuous Integration and Deployment Best Practices on AWS
•
5 likes•1,620 views
Continuous Integration and Deployment Best Practices on AWS
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Continuous Integration and Deployment Best Practices on AWS
Similar to Continuous Integration and Deployment Best Practices on AWS (20)
More from Danilo Poccia
More from Danilo Poccia (20)
Recently uploaded
Recently uploaded (20)
Continuous Integration and Deployment Best Practices on AWS
- 1. CONTINUOUS INTEGRATION AND DEPLOYMENT BEST PRACTICES ON AWS Danilo Poccia – Solutions Architect
- 2. Compute & Networking Storage & Content Delivery AWS Global Infrastucture Database App Services Analytics Deployment & Management API API API
- 3. API API API Regions Availability Zones Edge Locations EC2 VPC Route 53 Direct Connect S3 Glacier CloudFront Storage Gateway RDS DynamoDB ElastiCache Redshift SQS SNS SWF SES AppStream CloudSearch Elastic Transcoder IAM CloudTrail CloudWatch CloudFormation Elastic Beanstalk OpsWorks EMR Data Pipeline Kinesis WorkSpaces
- 4. Regions Availability Zones Edge Locations S3 Glacier CloudFront Storage Gateway RDS DynamoDB ElastiCache Redshift SQS SNS SWF SES AppStream CloudSearch Elastic Transcoder IAM CloudTrail CloudWatch CloudFormation Elastic Beanstalk OpsWorks EMR Data Pipeline Kinesis EC2 VPC Route 53 Direct Connect WorkSpaces
- 5. Regions Availability Zones Edge Locations EC2 VPC Route 53 DirectConnect S3 Glacier CloudFront Storage Gateway RDS DynamoDB ElastiCache RedshiftSQS SNS SWF SES AppStream CloudSearch ElasticTranscoder IAM CloudTrail CloudWatch CloudFormation Elastic Beanstalk OpsWorks EMR Data Pipeline Kinesis WorkSpaces
- 7. BUILDING BLOCKS TO IMPLEMENT YOUR IDEA
- 9. FILE SYNC
- 10. DATA ANALYSIS
- 11. DATA ANALYSIS COSTS < VALUE
- 14. 2013 280 2011 82 2010 61 2009 48 2008 24 2007 9 Amazon FPS Red Hat EC2 SimpleDB CloudFront EBS Availability Zones Elastic IPs Relational Database Service Virtual Private Cloud Elastic Map Reduce Auto Scaling Reserved Instances Elastic Load Balancer Simple Notification Service Route 53 RDS Multi-‐AZ Singapore Region Identity Access Management Cluster Instances Elastic Beanstalk Simple Email Service CloudFormation RDS for Oracle ElastiCache Redshift Kinesis WorkSpaces Route 53 Health Check OpsWorks CloudTrail number of released features, sample services described 2012 159Glacier DynamoDB Simple Workflow CloudSearch Storage Gateway Route 53 Latency Based Routing
- 15. NEW SERVICES + NEW FEATURES = NEW IDEAS
- 16. AMAZON KINESIS REAL-TIME PROCESSING OF STREAMING DATA AT MASSIVE SCALE
- 17. !Data! Sources! App.4% ! [Machine! Learning]! !!!!!!!!!!!!!!! ! ! ! AWS!Endpoint! App.1% % [Aggregate!&! De:Duplicate]! !Data! Sources! Data! Sources! !Data! Sources! App.2% % [Metric! Extrac=on]! S3 DynamoDB Redshift App.3% [Sliding! Window! Analysis]! !Data! Sources! Availability Zone Shard 1 Shard 2 Shard N Availability Zone Availability Zone
- 18. CLICKSTREAM ANALYTICS INTERNET OF THINGS METERING & BILLING EVENT PROCESSING SOCIAL NETWORKS LOG COLLECTION …
- 19. AMAZON SQS + AUTO SCALING + SPOT INSTANCES ! = FREEMIUM
- 22. EVEO nline
- 24. THE JOURNEY OF CODE TO PRODUCTION
- 27. LEAN MANUFACTURING Transport Inventory Motion Waiting Over Production Over Processing Defects Talent! & Skills
- 28. INFORMATION TECHNOLOGY Transport Inventory Motion Waiting Over Production Over Processing Defects Talent! & Skills
- 29. OPTIMIZE WITH AWS Transport Inventory Motion Waiting Over Production Over Processing Defects Talent! & Skills
- 30. OPTIMIZE WITH AWS OPPORTUNITY STRATEGY USE ADJUST
- 32. ! IF YOU CAN DO IT YOU CAN AUTOMATE IT
- 33. “Improving daily work is even more important than doing daily work.” ! ! Gene Kim, Kevin Behr, George Spafford, The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win
- 35. DEVELOPER
- 39. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER PICK TASKS
- 40. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SUBMIT CODE
- 41. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER SCHEDULE BUILD
- 42. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER AUTOMATIC BUILDS
- 43. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE FETCH
- 44. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER CODE QUALITY TESTS TEST RESULTS
- 45. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER BUILD OUTPUT
- 46. SOURCE CODE REPOSITORY PROJECT MANAGEMENT SERVER CONTINUOUS INTEGRATION SERVER DOCS BINARIES & PACKAGES
- 49. PAIN POINTS • UNIT TESTS INCOMPLETE • MOCK TESTS MAINTENANCE • EXPENSIVE TEST ENVIRONMENT • TEST ENVIRONMENT ≠ PRODUCTION • DEPLOYMENT CYCLES
- 50. “Complexity arises when the dependencies among the elements become important.” ! ! Scott E. Page, John H. Miller, Complex Adaptive Systems: An Introduction to Computational Models of Social Life
- 52. CLOSE THE FEEDBACK LOOP AS QUICKLY AS POSSIBLE
- 55. CONTINUOUS DEPLOYMENT SMALL, FREQUENT CHANGES CONSTANTLY INTEGRATING INTO PRODUCTION
- 56. KEY = ITERATION
- 57. ITERATION = MODIFY THE SYSTEM TO BETTER MEET THE EXPECTATIONS OF YOUR USERS
- 58. FEATURE SWITCH FEATURE FLIPPER FEATURE TOGGLE FEATURE FLAG CONDITIONAL FEATURE
- 59. // To totally enabled a feature ! $server_config[‘foo'] = ‘on’;! ! // To totally disabled a feature! $server_config['foo'] = ‘off’;! ! // Feature with variant turned on for everyone! $server_config['foo'] = 'blue_background';! ! // Enabled for 10% of regular users and all admins! $server_config[‘foo'] = array(! 'enabled' => 10,! 'admin' => 'on',! !); https://github.com/etsy/feature
- 60. !if (Feature::isEnabled('my_feature')) {! ! // do stuff! !}! ! ! !if (Feature::isEnabled('my_feature')) {! ! ! ! switch (Feature::variant('my_feature')) {! ! case 'foo':! ! // do stuff appropriate for the foo variant! ! break;! ! case 'bar':! ! // do stuff appropriate for the bar variant! ! break;! ! }! !} https://github.com/etsy/feature
- 61. A / B TESTING CONTENT
- 62. // Single-variant feature in 50/50 A/B test! ! $server_config['foo'] = array('enabled' => 50);! ! // Multi-variant feature in A/B test! // with 20% of users seeing each variant! // (and 40% left in control group)! ! $server_config['foo'] = array(! ! 'enabled' => array(! ! 'blue_background' => 20,! ! 'orange_background' => 20,! ! 'pink_background' => 20,! ! ),! !); https://github.com/etsy/feature
- 64. AWS OPSWORKS MODEL AND MANAGE THE ENTIRE APPLICATION
- 66. 14 BILLION REQUESTS/MONTH 50 000 DATABASE UPDATES / SEC NO CACHE C A SE STU D Y
- 71. A / B TESTING NOT JUST FOR CONTENT
- 79. AWS ELASTIC BEANSTALK QUICKLY DEPLOY AND MANAGE APPLICATIONS
- 82. Production Environment Test Environment master branch test branch Git Repository git commit git aws.push git aws.push Swap URLs
- 83. AMAZON EC2 API / SDK / CLI
- 85. for ip in $(! ! aws ec2 describe-instances! --filter Name=instance-state-name,Values=running,! Name=tag:aws:autoscaling:groupName,Values=XXX! --query Reservations[].Instances[].PublicDnsName! --output text! ! ); do ssh -t ec2-user@$ip "/app/update.sh"; done! !
- 87. DEPLOYMENTS AT AMAZON.COM ! ! ! 11.6s Mean time between deployments (weekday) 1,079 Max number of deployments in a single hour 10,000 Mean number of hosts simultaneously receiving a deployment 30,000 Max number of hosts simultaneously receiving a deployment
- 91. « Want to increase innovation? Lower the cost of failure » ! Joi Ito
- 92. CONTINUOUS INTEGRATION AND DEPLOYMENT BEST PRACTICES ON AWS Danilo Poccia – Solutions Architect