Chef in the cloud [dbccg]

Chef in the Cloud
Speaker:

Joshua Timberman Technical Evangelist
‣ joshua@opscode.com
‣ @jtimberman
‣ www.opscode.com
Copyright © 2010 Opscode, Inc - All Rights Reserved 1
Monday, September 27, 2010

System administrator
Many environments
Opscode: Training, Services

Copyright © 2010 Opscode, Inc - All Rights Reserved
http://www.ﬂickr.com/photos/anotherphotograph/2100904507/sizes/o/ 2

Developers?
Systems Administrators?
“Business” people?

http://www.ﬂickr.com/photos/timyates/2854357446/sizes/l/


Cloud Infrastructure

Provisioning
Conﬁguration Management
System Integration


Provisioning
Nodes
lb01

web01
web02

db-master01
db-slave01
db-slave02

Conﬁguration Management

Roles
loadbalancer
webserver
dbmaster
dbslave


Systems Integration
Load Balancer
Recipes
Web Server Web Server
haproxy
apache2
myssql DB Master

DB Slave DB Slave

Disk Disk Disk


Infrastructure as Code

http://www.ﬂickr.com/photos/wonderlane/2306082998/


Infrastructure as Code is...
A technical
domain revolving
around building
and managing
infrastructure
programmatically

http://www.ﬂickr.com/photos/kwerfeldein/2634561264/sizes/o/

Enable the reconstruction of
the business from nothing
but a source code
repository, an application
data backup, and bare metal
resources.

http://www.brooklynstreetart.com/theBlog/wp-content/uploads/2008/12/swedish_chef_bork-sleeper-cell.jpg

At a High Level...

‣ A library for configuration management
‣ A configuration management system
‣ A systems integration platform
‣ An API for your entire Infrastructure

http://www.flickr.com/photos/asten/2159525309/sizes/l/


Chef Client runs on your
systems


Clients talk to a Chef
Server


RESTful API w/ JSON
responses


The Opscode Platform
is a hosted Chef Server
http://www.opscode.com


We call each system you
conﬁgure a Node
Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.ﬂickr.com/photos/peterrosbjerg/3913766224/ 17

Nodes have Attributes

Kernel info!
{
"kernel": {
"machine": "x86_64",
"name": "Darwin",
"os": "Darwin",
"version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010;
root:xnu-1504.7.4~1/RELEASE_I386",
"release": "10.4.0"
},
"platform_version": "10.6.4",
"platform": "mac_os_x",
"platform_build": "10F569",
"domain": "local",
Platform info!
"os": "darwin",
"current_user": "jtimberman",
"ohai_time": 1278602661.60043,
"os_version": "10.4.0",
"uptime": "18 days 17 hours 49 minutes 18 seconds",
"ipaddress": "10.13.37.116",
"hostname": "cider",
"fqdn": "cider.local",
Hostname and IP!
"uptime_seconds": 1619358
}


Nodes have a Run List
What Roles or Recipes to apply
in Order


Nodes have Roles

Copyright © 2010 Opscode, Inc - All Rights Reserved http://www.ﬂickr.com/photos/laenulfean/374398044/ 20

Roles have a Run List

What Roles or Recipes to apply
in Order


remote_file
link
cookbook_file
service
ruby_block
template
execute

Chef manages
Resources on Nodes
package bash git log
deploy
user http_request

Resources...
Declare a description of the state a part of the node should be in

‣ Have a type package "apache2" do
version "2.2.11-2ubuntu2.6"
action :install
‣ Have a name end

template "/etc/apache2/apache2.conf" do
‣ Have parameters source "apache2.conf.erb"
owner "root"
‣ Take action to put the group "root"
mode 0644
resource in the action :create
declared state end

http://www.ﬂickr.com/photos/xiaming/382205902/sizes/l/

Resources take action
through Providers


Providers...
Know how to actually perform the actions speciﬁed by a resource.

Apt, Yum, Rubygems,
Multiple providers
per resource type.
Portage, Macports,
FreeBSD Ports, etc.

http://www.ﬂickr.com/photos/affableslinky/562950216/


Resources

Platform

Provider
http://www.ﬂickr.com/photos/acurbelo/2628837104/sizes/o/


Recipes are lists of
Resources

http://www.ﬂickr.com/photos/roadsidepictures/2478953342/sizes/o/

Recipes...
Apply resources in the order they are speciﬁed

package "apache2" do
version "2.2.11-2ubuntu2.6"
action :install
1
‣ Evaluates resources in [
end
the order they appear "package[apache2]",
"template[/etc/apache2/apache2.conf]"
template "/etc/apache2/apache2.conf" do
] source "apache2.conf.erb"
‣ Adds each resource to owner "root"
the Resource Collection group "root"
mode 0644
action :create
2
end

http://www.ﬂickr.com/photos/roadsidepictures/2478953342/sizes/o/


Order Matters

http://www.infrastructures.org/papers/turing/turing.html


Cookbooks are
packages for Recipes


Common Cookbook Components

recipes/
default.rb

ﬁles/
templates/
attributes/
default.rb

metadata.rb


Cookbooks are
shareable!

cookbooks.opscode.com

Data bags store
arbitrary data


A user data bag item...
% knife data bag show users jtimberman
{
"comment": "Joshua Timberman",
"groups": "sysadmin",
"ssh_keys": "ssh-rsa SUPERSEKRATS jtimberman@cider",
"files": {
".zshrc": {
"mode": "0644",
"source": "dot-zshrc"
},
".vimrc": {
"mode": "0644",
"source": "dot-vimrc"
}
},
"id": "jtimberman",
"uid": 7004,
"shell": "/usr/bin/zsh",
"openid": "http://jtimberman.myopenid.com/"
}

Copyright © 2010 Opscode, Inc - All Rights Reserved


sysadmins = search(:users, 'groups:sysadmin')

sysadminss.each do |u|
user u['id'] do
uid u['id']
shell u['shell']
comment u['comment']
supports :manage_home => true

Data bags make recipes home "/home/#{u['id']}"
end

awesome-r (that’s
directory "/home/#{u['id']}/.ssh" do
owner u['id']
group u['id']
mode 0700

totally a word)
end

template "/home/#{u['id']}/.ssh/authorized_keys" do
source "authorized_keys.erb"
owner u['id']
group u['id']
mode 0600
variables :ssh_keys => u['ssh_keys']
end
end

Nodes, Roles, Data
Bags are Searchable

% knife search node “role:webserver”

search(:users, “group:sysadmins”)


Automating the Cloud
with Chef

http://www.ﬂickr.com/photos/46183897@N00/3442880227/sizes/l/

lb1

web1 web2


Command-line API
utility, Knife

http://www.ﬂickr.com/photos/myklroventine/3474391066/

Fog

EC2
Rackspace/Openstack
Terremark/vcloud
Slicehost
Others...?


Add your Cloud
credentials to knife.rb
vi ~/chef-repo/.chef/knife.rb

# Cloud credentials
knife[:aws_access_key_id] = ENV['AWS_ACCESS_KEY_ID']
knife[:aws_secret_access_key] = ENV['AWS_SECRET_ACCESS_KEY']


Download some
cookbooks
% knife cookbook site vendor haproxy
INFO: Downloading haproxy from the cookbooks site at version 0.7.0
...
INFO: Merging changes from haproxy version 0.7.0.
...
INFO: Cookbook haproxy version 0.7.0 successfully vendored!

% knife cookbook site vendor apache2
...
INFO: Cookbook apache2 version 0.12.3 successfully vendored!


Upload Cookbooks!
knife cookbook upload -a

These run as root, kids.
Let’s not blindly trust the upstream too much!


Build some roles
% vi roles/lb.rb

name "lb"
description "Load Balancer"
run_list(
"recipe[haproxy]"
)

% vi roles/webserver.rb

name "webserver"
description "Systems that serve HTTP traffic"
run_list(
"recipe[apache2]"
)
default_attributes(
"apache2" => {
"listen_ports" => [ "80" ]
}
)

Upload Roles

% knife role from file lb.rb
WARN: HTTP Request Returned 404 Not Found: Cannot load role lb
WARN: Updated Role lb!

% knife role from file webserver.rb
WARN: HTTP Request Returned 404 Not Found: Cannot load role webserver
WARN: Updated Role webserver!


Launch a new Web
Server and Load
Balancer
knife ec2 server create ‘role[webserver]’

knife ec2 server create ‘role[lb]’


knife ec2 server create

Create EC2 instance via API
Retrieve local configuration
SSH to instance
‣ Write chef configuration and authentication
‣ Install Ruby and Chef
‣ Run Chef with specified run list


Chef runs on your new
server
sudo chef-client

INFO: Starting Chef Run
...
INFO: Chef Run complete in 211.852033 seconds

Automatically.

cookbooks/haproxy/recipes/default.rb
webservers = search(:node, "role:webserver")

template "/etc/haproxy/haproxy.cfg" do
source "haproxy.cfg.erb"
owner "root"
group "root"
mode 0644
variables(:webservers => webservers)
notifies :restart, resources(:service => "haproxy")
end

cookbooks/haproxy/templates/default/haproxy.cfg.erb
listen application 0.0.0.0:80
balance roundrobin
<% @webservers.each do |n| -%>
server <%= n[:hostname] %> <%= n[:ipaddress] %>:80 weight 1 maxconn
50 check
<% end -%>

cookbooks/apache2/templates/default/index.html.erb
<html>
<head>
<title>Welcome to <%= node[:hostname] %></title>
</head>
<body>
You have reached:
<ul>
<li>FQDN: <%= node[:fqdn] %></title>
<li>Public FQDN: <%= node[:ec2][:public_hostname]%></id>
<li>IP Address: <%= node[:ipaddress] %></id>
<li>Public IP: <%= node[:ec2][:public_ipv4] %></id>
<li>Platform: <%= node[:platform] %></id>
<li>Plaform Version: <%= node[:platform_version] %></id>
<li>Run List: <%= node.run_list %></id>
</ul>
</body>
</html>

Lessons Learned


AMIs

‣ Amazon Machine Images
‣ Use one AMI with JEOS for each size
‣ Beware of Image Sprawl
‣ Rebundle for long-running installation

http://www.ﬂickr.com/photos/wwworks/3271208324/sizes/l/

Instances

‣ c1.medium is best bang for the $
‣ User data can inform conﬁguration
‣ Use SSH key only access
‣ Allow SSH in default security group


Instances

‣ Internal and external network info is not
unique
‣ Only trust the instance-id


Storage

‣ EBS
‣ Persistent storage
‣ Trivial to snapshot
‣ Snapshots can create new devices


Elastic IP Addresses

‣ Static public routable addresses
‣ Easy and fast to re-assign
‣ Just an API call away


Resources/Questions

www.opscode.com/chef
IRC and Mailing lists
‣ irc.freenode.net #chef
‣ lists.opscode.com

Twitter:
‣ @opscode, #opschef
‣ @jtimberman

Questions?


Chef in the cloud [dbccg]

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (13)

Semelhante a Chef in the cloud [dbccg]

Semelhante a Chef in the cloud [dbccg] (20)

Último

Último (20)

Chef in the cloud [dbccg]