Introduction to Chef for Cloud Infrastructure

Introduction to Chef
SCALE Build a Cloud Day

matt@opscode.com
www.opscode.com
@mattray

Congratulations!!!

• U has a cloud
• Now what?

http://www.ﬂickr.com/photos/ian_munroe/4758240536/

APIs are awesome!

• You can provision
compute resources in
seconds
• You can provision
storage resources in
seconds
• That’s cool.

http://www.ﬂickr.com/photos/jdhancock/3634246981/

Chef can help with that
• knife ec2 server • knife cloudstack
create server create
• knife rackspace • knife kvm server
server create create
• knife terremark • knife vsphere
server create server create
• knife voxel • knife eucalyptus
• knife gandi • knife openstack
http://www.ﬂickr.com/photos/kyz/3122499444/

But then what?

http://www.ﬂickr.com/photos/doctorow/2698336843

See Nodes

Application Server

Application Database

See Nodes Grow

Application Server

Application Databases

See Nodes Grow

Application Servers


See Nodes Grow
Load Balancer

Application Servers


See Nodes Grow
Load Balancers

Application Servers


See Nodes Grow
Load Balancers

Application Servers

Application Database Cache


Tied together with Config
Load Balancers

Application Servers



Infrastructure is a Snowflake
Load Balancers

Application Servers


Floating IP?


Evolving Complexity

Load Balancers
ApplicationCache

Application Servers
NoSQL Database Cache

Database Slaves

Database

Complexity Grows Quickly
DC2

DC1

DC3

And it Continues to Evolve

http://www.ﬂickr.com/photos/16339684@N00/2681435235/

Golden Images are not the answer

• Gold is heavy
• Hard to transport
• Hard to mold
• Easy to lose
configuration detail

http://www.ﬂickr.com/photos/garysoup/2977173063/

Typical Boring Infrastructure

Graphite Nagios

Jboss App

Memcache

Postgres Slaves

Postgres Master

New Compliance Mandate

Graphite Nagios

Jboss App

• Move SSH off port 22
• Lets put it on 2022 Memcache

Postgres Slaves

Postgres Master

6 Golden Image Updates

Graphite 1 2 Nagios

3 Jboss App

• edit /etc/ssh/sshd_config 4 Memcache

5 Postgres Slaves

6 Postgres Master

12 Instance Replacements

Graphite 1 2 Nagios

3 4 5 6 7 Jboss App
• Delete, launch 8
• Repeat 9 Memcache
• Typically manually
10 11 Postgres Slaves

12 Postgres Master

In a Maintenance Window

Graphite 1 2 Nagios

3 4 5 6 7 Jboss App

• Don't break anything! 8 9 Memcache

5
10 11 Postgres Slaves

12 Postgres Master

With Different IP Addresses?

Graphite Nagios

Jboss App

• Invalid Configs
• Bob just got fired :( Memcache

Postgres Slaves

Postgres Master

Configuration Desperation

http://www.ﬂickr.com/photos/francoforeshock/5716969942/

Configuration Management and
Automated Systems Integration
is the Answer

http://www.ﬂickr.com/photos/philliecasablanca/3354734116/

Chef Solves This Problem

• But you already
guessed that, didn’t
you?

Collections of Resources
• Networking • Routes
• Users
• Files • Groups
• Directories • Tasks
• Symlinks • Packages
• Mounts • Software
• Services
• Configurations
• Other Stuff
http://www.ﬂickr.com/photos/stevekeys/3123167585/

Acting in Concert

http://www.ﬂickr.com/photos/glowjangles/4081048126/

To Provide a Service

http://www.ﬂickr.com/photos/28309157@N08/3743455858/

Chef is Infrastructure as Code

• Programmatically
provision and configure
• Treat like any other code
base
• Reconstruct business from
code repository, data
backup, and bare metal
resources.

http://www.ﬂickr.com/photos/louisb/4555295187/

Declarative Interface to Resources

• Define policy
• Say what, not how
• Pull not Push

http://www.ﬂickr.com/photos/bixentro/2591838509/

That looks like this

extra_packages = case node['platform']
when "ubuntu","debian"
%w{
ruby1.8
ruby1.8-dev
rdoc1.8
ri1.8
libopenssl-ruby
}
end
extra_packages.each do |pkg|
package pkg do
action :install
end
end

Or this
search(:users, '*:*') do |u|
user u['id'] do
uid u['uid']
shell u['shell']
home "/home/#{u['id']}"
end
directory "#{home_dir}/.ssh" do
owner u['id']
group u['gid']
mode "0700"
end
template "#{home_dir}/.ssh/authorized_keys" do
source "authorized_keys.erb"
owner u['id']
group u['id']
mode "0600"
variables :ssh_keys => u['ssh_keys']
end
end

Recipes and Cookbooks

• Recipes are collections of
Resources
• Cookbooks contain
recipes, templates, files,
custom resources, etc
• Code re-use and
modularity
• Hundreds already on
Community.opscode.com

http://www.flickr.com/photos/shutterhacks/4474421855/

Nodes
• Chef-Client generates
configurations directly
on nodes
• Reduce management
complexity through
abstraction
• Store the configuration
of your programs in
version control

http://www.flickr.com/photos/ssoosay/5126146763/

Upload your infrastructure
knife cookbook upload apt
knife cookbook upload chef-client
knife cookbook upload java
knife cookbook upload jpackage
knife cookbook upload ntp
knife cookbook upload sudo
knife cookbook upload tomcat
knife cookbook upload users
knife cookbook upload sample
knife role from file base-cloud.rb
knife role from file tc.rb
knife role from file sample.rb
knife data bag create users
knife data bag from file users mray.json

Build it somewhere
#EC2
knife ec2 server create -S mray -i ~/.ssh/mray.pem -x ubuntu -G default -I ami-a7a97dce
-f m1.small -d omnibus -r 'role[base-cloud],role[tc],role[sample]'
#Rackspace
knife rackspace server create --image 110 --flavor 2 -i ~/.ssh/mray.pem -d omnibus -r
'role[base-cloud],role[tc],role[sample]'
#CloudStack
knife cs server create -S "small instance" -T "CentOS 5.5(64-bit) no GUI (KVM)" -i
~/.ssh/mray.pem -d omnibus -r 'role[base-cloud],role[tc],role[sample]'
#Ubuntu Linux
knife bootstrap test.lab -r 'role[webserver]' -i ~/.ssh/mray.pem -x ubuntu --sudo -d
omnibus -r 'role[base-cloud],role[tc],role[sample]'

Search

• Search for nodes
with Roles
• Find configuration
data

• IP addresses
• Hostnames
• FQDNs

http://www.ﬂickr.com/photos/kathycsus/2686772625

Pass results into Templates

pool_members = search("node","role:webserver”)

template "/etc/haproxy/haproxy.cfg" do
source "haproxy-app_lb.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members.uniq
notifies :restart, "service[haproxy]"
end

Pass results into Templates

# Set up application listeners here.
listen application 0.0.0.0:80
balance roundrobin
<% @pool_members.each do |member| -%>
server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check
<% end -%>
<% if node["haproxy"]["enable_admin"] -%>
listen admin 0.0.0.0:22002
mode http
stats uri /
<% end -%>

So when this
Graphite Nagios

Jboss App

Memcache

Postgres Slaves

Postgres Master

Becomes this
Graphite Nagios

Jboss App

Memcache

Postgres Slaves

Postgres Master

Updates can be automatic
Graphite Nagios

Jboss App

Memcache

Postgres Slaves

Postgres Master

Count the resources
• Load balancer config
Graphite Nagios • Nagios host ping
• Nagios host ssh
Jboss App • Nagios host HTTP
• Nagios host app health
Memcache • Graphite CPU
• Graphite Memory
Postgres Slaves • Graphite Disk
• Graphite SNMP
• Memcache firewall
• 12+ resource changes for 1 node addition • Postgres firewall
Postgres authZ config

CLONING CANNOT COPE WITH THIS

• Chef can.

http://www.ﬂickr.com/photos/evelynishere/2798236471/

Build anything
• Simple internal applications
• Complex external applications
• Workstations
• Hadoop clusters
• IaaS infrastructure
• PaaS infrastructure
• SaaS applications
• Storage systems
• You name it
http://www.ﬂickr.com/photos/hyku/245010680/

And manage it simply
• Automatically
reconfigure
everything
• Linux, Windows,
Unixes, BSDs
• Load balancers
• Metrics collection
systems
• Monitoring systems
• Cloud migrations
become trivial
http://www.ﬂickr.com/photos/helico/404640681/

The Chef Community

• Apache License, Version 2.0
• 550+ Individual contributors
• 100+ Corporate contributors
• Dell, Rackspace,VMware, RightScale,
Heroku, and many more
• Nearly 400 cookbooks
• http://community.opscode.com

Questions?

http://www.ﬂickr.com/photos/mrchippy/443960682/

Thanks!

matt@opscode.com
www.opscode.com

Introduction to Chef for Cloud Infrastructure

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (8)

Semelhante a Introduction to Chef for Cloud Infrastructure

Semelhante a Introduction to Chef for Cloud Infrastructure (20)

Mais de Chef Software, Inc.

Mais de Chef Software, Inc. (20)

Último

Último (20)

Introduction to Chef for Cloud Infrastructure