Deploying systems using AWS DevOps tools
You've heard a lot about DevOps, but have you ever wondered which tools to use to deploy your systems? Join Karl Schwirz and Matt Parr from Slalom Consulting as they walk through a code pipeline deployment on AWS. In this MassTLC DevOps session, Matt and Karl will walk through a real-world application deployment using CloudFormation, CodeDeploy, CodePipeline and Chef.
8. 8
DevOps is about tools and a process
You want to break down the barriers that traditionally exist which separate
the development and operations teams.
“Breaking down silos and bringing people together is
the MOST IMPORTANT part of DevOps.”
- GearStream Blog
Aligning your teams to work together to establish a
process which enables them to drive toward the
same goal
14. Staging
Code Deploy
Production
Dev
Application Deployment Groups
Revisions
S3 EC2
Code Deploy
“AWS CodeDeploy is a service that automates code deployments to any instance, including Amazon EC2 instances and
instances running on-premises.“ – Read More
23. Configuration Management
Configuration management (CM) : a systems engineering process for
establishing and maintaining consistency of a product's performance, functional
and physical attributes with its requirements, design and operational information
throughout its life.
24. Case Study : Improvement Opportunities
Entirely manual
Multiple teams involved
Scattered and undocumented process
Poor knowledge transfer / knowledge gaps
Multiple bottlenecks and single points of failure
Inconsistent and not repeatable
Time consuming
29. Well-established support community through
Puppet Labs.
Mature interface that is intuitive and runs on
nearly every OS.
Simple installation and setup.
Strong reporting capabilities.
Puppet
Ruby knowledge required for advanced tasks
Pure-ruby versions are being scaled back with more
focus on Puppet’s customized DSL
Not focused on simplicity
Code base can grow large and unwieldy
Steep learning curve
Model-driven approach means less control
compared to code-driven approaches.
30. Rich collection of modules and configuration
recipes.
Code-driven approach gives you more control
and flexibility over your configurations.
Being centered around Git gives it strong
version control capabilities.
Ease of install when it comes to it’s
components.
Chef
Learning curve is steep if you’re not already
familiar with Ruby and procedural coding.
It’s not a simple tool, which can lead to large
code bases and complicated environments.
Doesn’t support push functionality.
31.
32. Utilize code to manage configurations and
automate provisioning of infrastructure in
addition to deployments
35. CloudFormation
Collection of related AWS resources
Visualize your infrastructure
Predictable
Orderly
Efficient
Customizable
36. CloudFormation
Version controlled infrastructure – your hardware is scripted!
Nothing is forever!
Ease of integration
Increased organizational efficiency
So we’re here to talk about DevOps tonight? To talk about tools on the AWS stack that help our teams when implementing DevOps solutions.
Quick show of hands; who’s been to a DevOps talk before?
Everyone is talking about DevOps… Including us. And it’s great. The more people in the conversation the better.
It’s not just small boutiques, either.
Netflix – forward thinkers in the space.
Hubspot – Local to Boston. (Now nation wide LA – BOS)
SpaceX – Big Enterprise taking on DevOps
Facebook
Spotify
Etsy
Disney
Chaos Monkey – The name comes from the idea of releasing a wild monkey with a weapon on your production data center and letting it cause havoc on random infrastructure and using this as an opportunity to learn the weaknesses of the system and continuously improve your ability to absorb and recover from failures (cattle vs. pets)
Before I go down the technical road of cool tools, we should remember. A clearly defined DevOps process is just as important as picking the right tool for the job.
Agile project management is a major tenant of DevOps. Getting an agile team together to agree and subscribe to a process will create an environment of collaboration and by default promote a feedback loop that will allow you to refine the process.
When you’re looking at tools, think about how much control you want think about how much control you have over your services.
Something like Cloud formation gives you granular control over the particulars of your systems
And on the other end, something like elastic beanstalk is more convenient, but you’re relying on AWS to provision and configure your system.
Icons across the top with descriptions of services
Continuous integration – merging code early and often
Continuous delivery – the process which code goes from Dev to Prod
Fully managed
Assumes you already have setup EC2 servers or on prem
Repeated deployments to existing application fleet
Configure Deployment Groups
Versioned Deployments
Automate scripts to execute during each deployment
Can be used with existing physical servers
Hooks - These steps are AWS defined
These steps are AWS defined
You might hear the term Continuous Delivery applied here
In a world without Config Management, amongst other difficulties….
First and foremost you’re following long procedural checklists
More people = more potential for a breakdown in the pipeline
Confusion and frustration
The ability to clearly define your infrastructure to fit it's landscape.
“a build should be a unit of code with configuration outside of it.”
Environments, roles, dependencies, etc
Eliminate manual changes, inconsistent state of resources, and human error.
Your infrastructure is version controlled - interacts with Git, etc Your configurations should endure the same rigorous testing and QA as your code.
Organized view of infrastructure past, present, and future.
Your configurations should endure the same testing and QA as your code.
There are many great options out there. Choose the best one that fits your needs.
Talk about tools – ansible, salt, chef, puppet, fabric
Tools – chef, puppet, ansible, salt, fabric. – each using different methods
Loads of online documentation, examples, etc. – you’re not alone.
There are "hooks", plugins, and native solutions available – go into windows specific tools, aws-s3, etc.
When you really get down to it – while they may all take different paths, they arrive at the same goal. You are managing your infrastructure and systems
Talk about plugins and addons – specific example is chef windows cookbook
Loads of easy to find information and documentation for most tools – chefdocs/puppetdocs examples
Talk about user community and contributors
You are not alone – chances are you aren’t the first to have issues, obstacles, and ideas – seek and you shall find
Puppet is a good choice if stability and maturity are key factors for you.
It’s good for large enterprises with a heterogeneous environment and range of skills on the DevOps team.
Puppet is one of the long standing tools in the full-fledged configuration management space.
It’s an open source tool, but given how long it’s been around, it has been well vetted and deployed in some of the biggest and most demanding environments.
Puppet is based in Ruby, but uses a customized Domain Scripting Language (DSL) closer to JSON for working within it.
It runs as a master-client setup and uses a model-driven approach.
The Puppet code design works as a list of dependencies, which can make things easier or more confusing, depending on your setup.
Puppet is a good choice if stability and maturity are key factors for you.
It’s good for large enterprises with a heterogeneous environment and range of skills on the DevOps team.
Puppet is one of the long standing tools in the full-fledged configuration management space.
It’s an open source tool, but given how long it’s been around, it has been well vetted and deployed in some of the biggest and most demanding environments.
Puppet is based in Ruby, but uses a customized Domain Scripting Language (DSL) closer to JSON for working within it.
It runs as a master-client setup and uses a model-driven approach.
The Puppet code design works as a list of dependencies, which can make things easier or more confusing, depending on your setup.
Reduced learning curve with descriptive languages such as those used by Chef, Puppet, and Ansible.
Flexibility allows for use with other languages (python, ruby, etc) as well as the ability to leverage vendor APIs.
IaC is an organized and concise representation of your infrastructure's configuration.
Code should follow the same rigorous testing procedures and QA as your other code
IaC can be used to achieve full orchestration of the entire deployment process as well as the automation of all necessary tasks in that cycle.
CloudFormation is an easy way to create and manage a collection of related AWS resources.
Provisioning and updating “stacks” and resources becomes a predictable, orderly, and efficient process.
Flexible templates allow for customization using parameters, forming relationships between resources, and scripting.
Provides the ability to visualize your infrastructure.
CloudFormation provides a means to version control your infrastructure using your organization’s tool of choice.
CloudFormation stacks can be launched and/or terminated via the AWS Management Console, AWS API, or the AWS command line interface. – cows and pigs
With the use of custom scripting, CloudFormation can be integrated with configuration management tools.