AWS OpsWorks is a solution for managing applications of any scale or complexity on the AWS cloud. Accelerate your use of OpsWorks by learning how to use several of its operational features in this Zero to Sixty session. It starts with a demo of the OpsWorks main workflows—manage and configure instances, create and deploy apps, monitoring, and security. BeachMint will explain how they set up OpsWorks as part of their continuous deployment pipeline. The session finishes off by explaining how to use the OpsWorks API and Chef recipes to automate standard operating procedures. Demos and code samples are available to all session attendees.
Are you new to AWS OpsWorks? Get up to speed for this session by first completing the 60-minute Introduction to AWS OpsWorks lab in the Self-Paced Hands-On Lab Lounge. It will lead you through all major functions of the service with a fun example.
2. AWS OpsWorks
Helps you to model your entire application from
load balancers to databases starting from
templates for common technologies or building
your own.
You have full control of deployments, scaling,
monitoring, and automation of each component.
3. AWS Application Management Services
Higher-level services
Elastic Beanstalk
Convenience
OpsWorks
Do it yourself
CloudFormation
EC2
Control
4. The Heart of AWS OpsWorks
Agent on each
EC2 instance
talks with
OpsWorks
5. The Heart of AWS OpsWorks
Agent on each
EC2 instance
Understands a set of commands
that are triggered by OpsWorks.
The agent then runs Chef solo.
6. The Heart of AWS OpsWorks
Agent on each
EC2 instance
Understands a set of commands
that are triggered by OpsWorks.
The agent then runs Chef solo.
7. The Heart of AWS OpsWorks
Agent on each
EC2 instance
Understands a set of commands
that are triggered by OpsWorks.
The agent then runs Chef solo.
13. Recap of Demo
•
•
•
•
•
•
Created a new Stack from scratch
Created an app and database layer
Started instances in different Availability Zones
Configured Auto Scaling
Load balanced by Elastic Load Balancing
Deployed an application
16. •
•
•
•
Founded in 2011
Next generation social commerce company
Parent to: JewelMint, ShoeMint, StyleMint, & IntiMint
Over 6 million registered customers
21. Confidence to Release Every Day
•
•
•
•
Unit tests
Integration tests
Load testing
Manual testing
22. What it Looks Like in AWS OpsWorks
Environments are represented as
stacks
Our different verticals are
represented as separate layers within
each stack
23. Architecture Repeated in Each Environment
IntiMint
ShoeMint
StyleMint
Stack
JavaScript widget
API
Checkout
RDS
Registration
Solr
Product
ElastiCache
Drupal
Social
Drools
Layers
JewelMint
25. AWS OpsWorks in Our CI Processes
• Get instances associated to a layer
• Update code based on gittag version stored in
custom JSON
• Target the instances and run Chef scripts
26. The Glue: Custom JSON
{
"environment":{
"env":"production"
},
"jewelmint":{
"gittag":"20131008d_release" },
"intimint":{
"gittag":"v3.7.0_release" },
"stylemint":{
"gittag":"v3.7.0_release" },
"shoemint":{
"gittag":"v3.7.0_release" },
"cdn":{
"media":{
"origin":"beachmint-origin-domain.com",
"domain":"beachmint-cloudfront.cloudfront.net"
}
},
"solr":{
"gittag":"master",
"server_name":"beachmint-solr-domain.com",
"nodes":{ "1":"node-ip-1",
"2":"node-ip-2",
"3":"node-ip-3" }
}
}
"environment":{
"env":"production”
},
Determines what
environment to be built
27. The Glue: Custom JSON
{
"environment":{
"env":"production"
},
"jewelmint":{
"gittag":"20131008d_release" },
"intimint":{
"gittag":"v3.7.0_release" },
"stylemint":{
"gittag":"v3.7.0_release" },
"shoemint":{
"gittag":"v3.7.0_release" },
"cdn":{
"media":{
"origin":"beachmint-origin-domain.com",
"domain":"beachmint-cloudfront.cloudfront.net"
}
},
"solr":{
"gittag":"master",
"server_name":"beachmint-solr-domain.com",
"nodes":{ "1":"node-ip-1",
"2":"node-ip-2",
"3":"node-ip-3" }
}
}
"jewelmint"{
"gittag":"20131008d_release" },
"intimint":{
"gittag":"v3.7.0_release" },
"stylemint":{
"gittag":"v3.7.0_release" },
"shoemint":{
"gittag":"v3.7.0_release" },
Code version to be
deployed to a layer
29. Commit
Build Stack
Jenkins post
commit hook
Get
instances
Update code
on instances
use AwsCommonAws;
$aws = Aws::factory('./config.php');
$opsWorks = $aws->get('OpsWorks');
//Get the Stack we are updating
$onlineInstanceIds = array();
$allInstances = $opsWorks->describeInstances(array('LayerId' => $layerId))>get("Instances");
foreach($allInstances as $instance) {
if ($instance['Status'] == 'online') {
$onlineInstanceIds[] = $instance['InstanceId'];
}
}
Run selenium
or unit tests
Y
Create new
tag
pass
Update git tag
on custom json
Update code
on instances
N
Stop
Load Stack
Run load
test
Ready for
QA
30. Commit
Build Stack
Jenkins post
commit hook
Get
instances
Update code
on instances
use AwsCommonAws;
$aws = Aws::factory('./config.php');
$opsWorks = $aws->get('OpsWorks');
$deployment = $opsWorks->createDeployment(array(
'StackId' => $stackId,
'Command' => array(
'Name' => 'execute_recipes',
'Args' => array(
'recipes' => $recipe
)
),
'InstanceIds' => $onlineInstanceIds
));
Run selenium
or unit tests
Y
Create new
tag
pass
Update git tag
on custom json
Update code
on instances
N
Stop
Load Stack
Run load
test
Ready for
QA
32. Commit
Build Stack
Jenkins post
commit hook
Get
instances
Update code
on instances
//Get the Stack’s custom JSON from OpsWorks
$customJsonArray = json_decode($theBuildStack[0]["CustomJson"]);
//Replace the gittag with the new tag
$customJsonArray->{"jewelmint"}->{"gittag"} = "thenewgittag";
//Convert back to string
$modifiedJson = json_encode($customJsonArray);
//Update Stack settings in OpsWorks
$updateRes = $opsWorks->updateStack(array("StackId" => "the-buildstackid","CustomJson" => $modifiedJson));
Run selenium
or unit tests
Y
Create new
tag
pass
Update git tag
on custom json
Update code
on instances
N
Stop
Load Stack
Run load
test
Ready for
QA
33. Commit
Build Stack
Jenkins post
commit hook
Get
instances
Update code
on instances
use AwsCommonAws;
$aws = Aws::factory('./config.php');
$opsWorks = $aws->get('OpsWorks');
$deployment = $opsWorks->createDeployment(array(
'StackId' => $stackId,
'Command' => array(
'Name' => 'execute_recipes',
'Args' => array(
'recipes' => $recipe
)
),
'InstanceIds' => $onlineInstanceIds
));
Run selenium
or unit tests
Y
Create new
tag
pass
Update git tag
on custom json
Update code
on instances
N
Stop
Load Stack
Run load
test
Ready for
QA
34. Path to Production
• QA and Product group determines what goes to
production
• Each release candidate has a tag associated
with it
• QA updates QA and Stage environments using a
tool based on the AWS SDK
50. Demo Recap
•
•
•
•
Instances set up alarms on CloudWatch
Alarms create notification in SNS
SNS publishes to SQS queue
Watcher instance polls queue and calls AWS
OpsWorks to execute recipe on instance
• The recipe runs a script to upload logs to S3
51. More Information about AWS OpsWorks
•
•
•
•
If not done yet, do the AWS OpsWorks lab!
Find us in the AWS Booth
Follow us on Twitter @AWSOpsWorks
Find us on YouTube
• AWS OpsWorks survey
http://tinyurl.com/OpsWorksSurvey2013
52. Other Talks During re:Invent
DMG304 - AWS OpsWorks Under the Hood
• Jonathan Weiss & Reza Spagnolo
• Thursday, Nov 14, 3:00 PM - 4:00 PM – Murano 3206
DMG305 - How Intuit Leveraged AWS OpsWorks
as the Engine of Our PaaS
• Capen Brinkley & Rick Mendes of Intuit, Inc.
• Thursday, Nov 14, 4:15 PM - 5:15 PM – Murano 3206
53. Please give us your feedback on this
presentation
DMG202
As a thank you, we will select prize
winners daily for completed surveys!
55. Recipes
• Default
Writes Ruby script to the instance and runs it
• create_alarm
Writes a Ruby script to create an alarm and ties it to the right SNS
topic
• send_logs
Writes a Ruby script to the instance that can pack and ship the logs
to S3
56. Watcher Code
queue = AWS::SQS.new.queues["<%= node[:watcher][:sqs][:url] %>"]
queue.poll(:initial_timeout => false) do |msg|
begin
alarm = JSON.parse(msg.body)['Subject'].start_with?('ALARM')
instance_id = JSON.parse((JSON.parse(msg.body)['Message']))['Trigger']
['Dimensions'].first['value']
if alarm
opsworks = AWS::OpsWorks.new.client
deployment = opsworks.create_deployment(
:stack_id => "<%= node[:opsworks][:stack][:id] %>",
:instance_ids => [instance_id],
:command => {
:name => 'execute_recipes',
:args => {'recipes' => <%= node[:watcher][:execute_recipes] %>}
},) end end end