Priming Your Teams For Microservice Deployment to the Cloud
•
7 gostaram•1,416 visualizações
You think of a great idea for a microservice and want to ship it to production as quickly as possible. Of course you'll need to create a Git repo with a codebase that reuses libraries you share with other services. And you'll want a build and a basic test suite. You'll want to deploy it to immutable servers using infrastructure as code that dev and ops can maintain. Centralised logging, monitoring, and HipChat notifications would also be great. Of course you'll want a load balancer and a CNAME that your other microservices can hit. You'd love to have blue-green deploys and the ability to deploy updates at any time through a Continuous Delivery pipeline. Phew! How long will it take to set all this up? A couple of days? A week? A month? What if you could do all of this within 30 minutes? And with a click of a button soon be receiving production traffic? Matt introduces "Primer", Expedia's microservice generation and deployment platform that enables rapid experimentation in the cloud, how it's caused unprecedented rates of learning, and explain tips and tricks on how to build one yourself with practical takeaways for everyone from the startup to the enterprise. Video: https://www.youtube.com/watch?v=Xy4EkaXyEs4 Meetup: http://www.meetup.com/Devops-Brisbane/events/225050723/
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Priming Your Teams For Microservice Deployment to the Cloud
Semelhante a Priming Your Teams For Microservice Deployment to the Cloud (20)
Último
Último (20)
Priming Your Teams For Microservice Deployment to the Cloud
- 1. Priming Your Teams for Microservice Deployment to the Cloud Matt Callanan linkedin.com/in/matthewcallanan @mcallana © 2015 Expedia, Inc. All rights reserved.
- 2. How Do You Create A Microservice For The Cloud?
- 3. Git Repo
- 11. Load Balancer
- 12. CNAME
- 15. Phew!
- 16. Opportunity Cost: Time Value of Information
- 17. “Primer”
- 18. DevOps Toolchain Based on devops toolchain by dto solutions: http://www.slideshare.net/AnthonyShortland/dto-chefconf2012 ApplicationInfrastructure Build Deploy Source Code Repo Artifact Repo Build Console Deployment Console Resource Model Infrastructure Manager Code Config Automation Environment Data Automation Config Packages Images
- 19. DevOps Toolchain Developer Toolchain Operations Toolchain
- 20. DevOps Toolchain Developer Toolchain Operations Toolchain
- 21. Dev – Starting from Scratch Single page app • Secured with LDAP • Choose: – App Name – Team Name – Notifications Email – HipChat Room – Technology Build triggers remote Jenkins job • Benefits – Easy to configure – Simple API – Versioned scripts – Centralisedlogs / results
- 22. Dev – App Types • dwsvc -‐ Java Dropwizard service • emr-‐datapipeline -‐ An Elastic MapReduce -‐ Data Pipeline service • spark-‐streaming -‐ A Spark Streaming App • expressjs -‐ A Node application that uses expressjs • php -‐ A PHP application • python -‐ A Python application • scalatra– Lightweight Scala app based off Sinatra • sinatra – Lightweight Ruby web service • springboot -‐ A SpringBoot service • storm-‐topology -‐ A storm topology service • ticker -‐ A Ticker (trend service) application • webmvc-‐ A customer facing Brand Expedia Spring MVC Web application • grails -‐ A Groovy-‐based web application framework for JVM • AWS Lambda -‐ Java based AWS Lamda implementation *each app type has it’s own GitHub repo for source, issues, etc
- 23. Dev – Generating App and Pipeline Jenkins Job – create-‐new-‐application • Creates new GitHub repo
- 24. Dev – Generating App and Pipeline Jenkins Job – create-‐new-‐application • Creates new GitHub repo GitHub • Promote open source model for teams • Use pull requests as mechanism for ensuring reviews / pairing • WebHooks for devs to tweak workflows
- 25. Dev – Generating App and Pipeline Jenkins Job – create-‐new-‐application • Creates new GitHub repo • Copies chosen technology template and pushes back to repo Tech Templates • Uses latest version of template • Replaces app names, package names, etc with your app name • Simple interface – each template codebase has clone.sh using cp and sed • Java-‐based templates use Maven Archetypes to generate codebase • Easy to extend and fork
- 26. Dev – Generating App and Pipeline Jenkins Job – create-‐new-‐application • Creates new GitHub repo • Copies chosen technology template and pushes back to repo • Generates Continuous Delivery pipeline in Jenkins Jenkins • Generates Jenkins build that is triggered by commits to master branch • Jenkins config via templated XML • Uses standard commands for target template (mvn, npm, etc) • Uploads zip to S3 • Triggers downstream deploy job passing Git SHA
- 27. Dev – Generating App and Pipeline Jenkins Job – create-‐new-‐application • Creates new GitHub repo • Copies chosen technology template and pushes back to repo • Generates Continuous Delivery pipeline in Jenkins • Sends email to user with details / links
- 28. DevOps Toolchain Developer Toolchain Operations Toolchain
- 29. DevOps Toolchain Developer Toolchain Operations Toolchain
- 30. Operations Pipeline • Immutable Servers • Chef Cookbooks • CloudFormation Templates • Deploy to immutable servers using infrastructure as code that dev and ops can maintain • Blue-‐green deploys • HipChat notifications • Centralised logging -‐ Splunk • Centralised monitoring -‐ Hubble
- 31. Chef Cookbooks and Config
- 32. Chef Cookbooks and Config roles/primer-‐demo-‐service.rb _______________________________________________________________________ name "primer-demo-service" description "Webapp for primer-demo-service" run_list "recipe[s3curl]","recipe[primer-demo-service]" override_attributes( "java" => { "jdk_name" => "jdk8_0_05", } )
- 33. Chef Cookbooks and Config cookbooks/primer-‐demo-‐service/recipes/default.rb _______________________________________________________________________ node.default[:springboot][:application_name] = 'primer-demo' node.default[:springboot][:role_name] = 'primer-demo-service' include_recipe "expedia-platform::springboot"
- 34. Chef Cookbooks and Config deployment_config/<env>/primer-‐demo-‐service/config.json { "instances": { "min_count": 1, "max_count": 2, "auto_scale": true, "zones": { "us-‐east-‐1": ["us-‐east-‐1d"], "us-‐west-‐2": ["us-‐west-‐2c"] }, "destroy_old": true, "removeOldFromElb": true }, "app_type": "springboot", "app_port": 8080, "isactive_support": { "enabled": true, "file": "/opt/primer-‐demo-‐service/active.txt" }, "source_repo": ".../primer-‐demo-‐service.git", "notify": { "hipchat": {"room": "Demo Room"}, "email": {"addresses": ["username@domain.com"]}, "s3": {"bucket": "..."} }, "team": "Exp-‐Ops-‐Wotif-‐Tech-‐Australia", "prime_role": "role-‐prime-‐web" }
- 35. Deploy Build Jenkins Chef knife AMI Route53 ELBCloudFormation ASG EC2 InstanceEC2 Instance
- 36. Deployment – Email Notification
- 38. Blue-‐Green Deploys Safely Releasing Updates
- 39. Separating Deploy & Release Deploy Test Release
- 40. Blue Green Deploys – Deploy
- 41. Deploy
- 43. Release
- 44. Blue Green Deploys – Release
- 45. Release – Email Notification
- 47. Centralised Logging – Splunk
- 48. Centralised Monitoring & Alerting
- 49. Hubble Dashboard
- 50. Hubble Dashboard – Graphene
- 51. Hopscotch – Transparent Traffic Router • Proxy built on Nginx (similar to Netflix Zuul) • expedia.com/xyz -‐> xyz.expedia.com • Faster to update rules than Akamai 45min turnaround • E.g. Register microservice with hopscotch in Singapore (20% of traffic) route 10% of Singapore traffic to cloud app, when happy, dial up to 100%, then roll out to Europe and USA • Nice way to move traffic out of data centre into AWS with no customer impact
- 52. Primer Cleanup
- 53. Cleanup • Deleting project – Delete Codebase (Git repo), Build • Delete Cloudformation stack • Delete unused cloudformation stacks -‐ ELB request count 1hr
- 54. Cleanup – Delete the Microservice
- 55. Cleanup – Email Confirmation
- 56. Cleanup – Delete Cloud Formation Stack
- 57. Cleanup – Auto Deletion of Unused Stack in Test Environment
- 58. Primer 2.0 Docker on Amazon ECS
- 59. DevOps Toolchain – Primer 2.0: Docker on Amazon ECS Inspired by devops toolchain by dto solutions: http://www.slideshare.net/AnthonyShortland/dto-chefconf2012 ApplicationInfrastructure Build Deploy Source Code Repo Artifact Repo Build Console Deployment Console Resource Model Infrastructure Manager Code Config Automation Environment Data Automation Config Packages Images gzip AMI git Jenkins JenkinsAmazon S3 Chef CloudFormationDocker Registry Docker image JSON Amazon ECS
- 60. Primer 2.0 – Docker on Amazon ECS • Using Amazon EC2 Container Service (ECS) • Deploy Primer applications as Docker containers to an ECS cluster • Deploys take 7mins (instead of 30mins) • No AMI creation, no CloudFormation • Cost Optimisation • Using AWS SDK: • Register ECS Container Definition -‐> Task Definition -‐> Service • Create ELB • Create CNAME • Blue/Green Deploys with Canary service
- 61. Primer 2.0 – Docker on ECS Test Environment
- 62. Primer 2.0 – Docker on ECS Production Environment
- 63. c3vis – Open Source • c3vis: “Cloud Container Cluster Visualiser” • Visualises container deploys and resource utilization on Amazon ECS clusters • Open Source Software from Expedia • github.com/ExpediaDotCom/c3vis
- 64. Results
- 65. Results • 800 Primer apps and growing • Culture of Experimentation – Freedom of choice – Keys to prod – trust • Enabling Innovation – Open Source model for tool • visible • accessible • easy to contribute • “I want to add Grails to this tool!”, “Help yourself!” • Lowering Cost – Counters the cost of learning legacy deployment processes – Generate microservice without thinking about it
- 66. Tips & Tricks Building Your Own “Primer”
- 67. Tips & Tricks • Don’t “Lift & Shift” your monolith • Rethink • Start with non-‐critical workflow apps • Don’t need to start heterogeneous (i.e. multiple app types) • Doesn't need to be in the cloud • Cloud has benefits like immutable infrastructure, scaling, automation • But can do the same thing in the data centre • Small interface • /isActive endpoint for LB healthcheck • /buildInfo endpoint for version • "Just do it" attitude • Open source model helped spread it across the organisation
- 68. Q & AThanks! Any Questions? Matt Callanan linkedin.com/in/matthewcallanan @mcallana
- 69. Image Attribution Image “Clock Time Lapse Video Download“ (https://youtu.be/NSuNpiW-‐LwI) by LifeBloodMarketing is licensed under “CC Free to Use” “Pipelines descending to Inveruglas Power Station” (http://www.geograph.org.uk/photo/2214366) is licensed under CC BY SA 2.0 (http://creativecommons.org/licenses/by-‐sa/2.0/) / Desaturated and cropped from original “The Future” (https://flic.kr/p/26YCn1) by Kristian Bjornard is licensed under CC BY SA 2.0 (https://creativecommons.org/licenses/by-‐sa/2.0/) “CTA Loop Junction” (https://commons.wikimedia.org/wiki/File:CTA_loop_junction.jpg) by Daniel Schwen is licensed under CC BY SA 3.0 (https://creativecommons.org/licenses/by-‐sa/3.0/) “Logging operations at Millmoor Rig” (http://bit.ly/1Nb20LS) by Walter Baxter is licensed under CC BY SA 2.0 (https://creativecommons.org/licenses/by-‐sa/2.0/) “Traffic Monitoring” (https://commons.wikimedia.org/wiki/File:Traffic_Monitoring.JPG) by Suryasuharman is licensed under CC BY SA 3.0 (https://creativecommons.org/licenses/by-‐sa/3.0/) “DNS logo” (https://commons.wikimedia.org/wiki/File:DNS_logo.jpg) by I laramide I is licensed under CC BY SA 3.0 (https://creativecommons.org/licenses/by-‐sa/3.0/) “Light bulb icon tips” (https://commons.wikimedia.org/wiki/File:Light_bulb_icon_tips.svg) by YassineMrabet is licensed under CC BY SA 4.0 (https://creativecommons.org/licenses/by-‐sa/4.0/) “Matrix-‐code-‐computer-‐pc-‐data” (https://pixabay.com/en/matrix-‐code-‐computer-‐pc-‐data-‐356024/) by Comfreak is licensed under CC ZERO (https://creativecommons.org/publicdomain/zero/1.0/) “Sample-‐color-‐blue-‐green” (https://pixabay.com/en/sample-‐color-‐blue-‐green-‐rubber-‐815141/ ) by LyraBelacqua-‐Sally is licensed under CC ZERO (https://creativecommons.org/publicdomain/zero/1.0/) “Fashion-‐wristwatch-‐time” (https://www.pexels.com/photo/fashion-‐wristwatch-‐time-‐watch-‐1252/) by SplitShire.com is licensed under CC ZERO (https://creativecommons.org/publicdomain/zero/1.0/) “Chat” (https://openclipart.org/detail/129049/chat) by Merlin2525 is licensed under unlimited-‐commercial-‐use (https://openclipart.org/unlimited-‐commercial-‐use-‐clipart) “scales” (https://openclipart.org/detail/24101/scales) by scott_kirkwood is licensed under unlimited-‐commercial-‐use (https://openclipart.org/unlimited-‐commercial-‐use-‐clipart) “Wooden Bookcase” (https://openclipart.org/detail/191463/wooden-‐bookcase) by sirgazil is licensed under unlimited-‐commercial-‐use (https://openclipart.org/unlimited-‐commercial-‐use-‐clipart) “Compiz GIT Repository” (https://flic.kr/p/Ssras) by -‐= Treviño =-‐ is licensed under BY NC SA 2.0 (https://creativecommons.org/licenses/by-‐nc-‐sa/2.0) “logs” (https://flic.kr/p/9F8tjX) by Rick Payette is licensed under CC BY NC ND 2.0 (https://creativecommons.org/licenses/by-‐nc-‐nd/2.0) Docker logo used according to https://www.docker.com/brand-‐guidelines