Cooking with Chef
Server management made easy

Me?
Ken Robertson
Senior Software Engineer at Involver
Involver builds a social media marketing platform
Specialize in our platform’s reliability,
performance, and scalability

Our Chef Usage
10 separate environments
120+ servers
All managed with Chef
One Operations Engineer, Two Developers

What is Chef?
Server management and configuration in Ruby
Developed by OpsCode
Adopted by Engine Yard, 37signals, and more
Apache License

Chef isn’t alone
CFEngine: http://cfengine.com/
Puppet: http://www.puppetlabs.com/
http://en.wikipedia.org/wiki/
Comparison_of_open_source_configuration_man
agement_software

Why use Chef?
Repeatable system provisioning
Manual tweaks are not repeatable
Ease scaling
Avoid vendor lock-in

Chef is Repeatable
Continuous configuration management
Ensure system compliance
Recovery from failures

Chef Flavors
chef-solo
Single instance
chef-server
Cluster, centrally managed

Chef’s Toolkit
Cookbooks
Recipes
Attributes

Recipes are
everywhere!
Open source - Engine Yard, 37signals, OpsCode
One offs for specific configurations
Approach with caution

Recipe ingredients:
Sub-recipes
Resources
Attributes
Definitions
Static resources
Templates

Resources

Resources - Examples
execute "some-descriptive-text" do
command "uptime"
end

Resources - Examples
link "/usr/local/bin/foo" do
to "/usr/src/foo-#{version}/bin/foo"
end

Resources - Examples
directory "/home/foo/apps/bar" do
owner "foobar"
group "foobar"
recursive true
end

Resources - Examples
package "mongodb"
version "1.2.3"
action :install
end

Resources - Examples
package "mongodb"
action :install, :upgrade
end

Resources - Examples
service "nginx" do
supports :status => false, :start => true,
:restart => true, :reload => true
action [ :enable, :start ]
end

Resources - Examples
cookbook_file "/etc/profile" do
owner "root"
group "root"
mode 644
source "profile"
end

Resources - Examples
cookbook_file "/etc/profile" do
owner "root"
group "root"
mode 644
source "http://safesite.com/files/profile"
end

Resources - Examples
template "/etc/hosts" do
owner "root"
group "root"
mode 644
source "hosts.erb"
variables(:one => 1, :two => 2)
end

Resources - Examples
cron "clear_tmp_files_older_than_a_day" do
hour 0
minute 0
user "root"
command "do_something"
end

Resources - Conditions
execute "install-rubygems-for-jruby" do
command %Q{
curl http://production.cf.rubygems.org/rubygems/
rubygems-1.3.7.tgz -O &&
tar xvzf rubygems-1.3.7.tgz &&
pushd rubygems-1.3.7 &&
jruby ./setup.rb &&
popd &&
rm rubygems-1.3.7.tgz &&
rm -r rubygems-1.3.7
}
only_if { %x{jruby -S gem --version}.chomp !=
'1.3.7' }
end

Results/Expectations
execute 'install passenger and nginx' do
command %Q{
wget -N http://site/file.tar.gz &&
tar -xvvf file.tar.gz &&
passenger-install-nginx-module ...
}
creates '/data/nginx/sbin/nginx'
end

Triggers
template "/etc/nginx/apps/#{params[:name]}.conf" do
source "#{params[:name]}.nginx.erb"
owner node[:user]
group node[:user]
mode 0644
variables(
:stage => params[:stage],
:name => params[:name]
)
notifies :reload, "service[nginx]", :delayed
end

Triggers
execute "mysql-create-database" do
...
action :nothing
end
template "/tmp/mysql-#{params[:name]}.sql" do
source "create-database.sql.erb"
variables(:params => params)
notifies :run,
resources(:execute => "mysql-create-database"),
:immediately
end

Attributes
Runtime configuration values
Define defaults
Pass in at runtime (as JSON)
Available through through the ‘node’ variable

Default Attributes
cookbooks/myrecipe/attributes/*.rb:
nginx_user "www-data"
nginx_port "80"
=> node[:nginx_user]
=> node[:nginx_port]
nginx { :user => 'www-data', :port => 80 }
=> node[:nginx][:user]
=> node[:nginx][:port]

Merging Attributes
default.mysql[:bindir] = '/usr/local/mysql',
default.mysql[:root] = '/data/mysql',
default.mysql[:uid] = 'mysql',
default.mysql[:gid] = 'mysql',
default.mysql[:group_name] = 'mysql',
default.mysql[:version] = '5.1.47'
node[:mysql][:version]

Runtime Attributes
{
"nginx_user": "www-data",
"nginx_port": 80,
"nginx": {
"user": "www-data",
"port": 80
}
}

Runtime Attributes
{
"mysql": {
"version": "5.1.47",
"config": {
"log_slave_updates": true,
"auto_increment_increment": "2"
}
}
}

Definitions
Mini-recipes
Repeatable blocks or sub-functions
Definitions sub-directory of recipe
cookbooks/myrecipe/definitions

Definitions
link "/usr/local/bin/foo" do
to "/usr/src/foo-#{version}/bin/foo"
end

Definitions
for db in node[:mysql][:databases] do
mysql_database db[:name] do
root_user node[:mysql][:root_user] || 'root'
root_password node[:mysql][:root_password]
dbuser db[:user] || db[:name]
dbpassword db[:password]
end
end

Definitions
define :mysql_database do
execute "mysql-create-database" do
...
action :nothing
end
template "/tmp/mysql-#{params[:name]}.sql" do
source "create-database.sql.erb"
variables(:params => params)
notifies :run,
"execute[mysql-create-database]",
:immediately
end
end

Definitions
define :nginx_site do
include_recipe "nginx"
template "/etc/nginx/apps/#{params[:name]}.conf" do
source "#{params[:name]}.nginx.erb"
owner node[:user]
group node[:user]
mode 0644
variables(
:stage => params[:stage],
:name => params[:name]
)
notifies :reload, "service[nginx]", :delayed
end
end

Recipe Gotchas
Idempotency

Idempotency
execute "install-jruby" do
command %Q{
curl http://urlto/#{version}/jruby-src-#{version}.tar.gz -O &&
tar xvzf jruby-src-#{version}.tar.gz &&
pushd jruby-#{version} &&
ant &&
popd &&
mv jruby-#{version} /usr &&
rm jruby-src-#{version}.tar.gz &&
ln -snf /usr/jruby-#{version}/bin/jruby /usr/local/bin/jruby
}
creates "/usr/jruby-#{version}"
end

Idempotency - Fixed
execute "install-jruby" do
command %Q{
curl http://urlto/#{version}/jruby-src-#{version}.tar.gz -O &&
tar xvzf jruby-src-#{version}.tar.gz &&
pushd jruby-#{version} &&
ant &&
popd &&
mv jruby-#{version} /usr &&
rm jruby-src-#{version}.tar.gz
}
creates "/usr/jruby-#{version}/bin/jruby"
end
link "/usr/local/bin/jruby" do
to "/usr/jruby-#{version}/bin/jruby"
end

Recipe Gotchas
Idempotency
Package sources

Recipe Gotchas
Idempotency
Package sources
Install vs upgrade

Install vs Upgrade
package “git-core” do
action :install
end
package “git-core” do
action :install, :upgrade
end

Recipe Gotchas
Idempotency
Package sources
Install vs upgrade
Attribute abuse

Recipe Gotchas
Idempotency
Package sources
Install vs upgrade
Attribute abuse
Cowboys and Homers

Homer
packages.each do |pkg|
package pkg
end
execute "Nuke existing installs" do
command "rm -rf /etc/tinydns /etc/dnscache"
end
...

Cowboys
One offs
Lack of testing
Manual, undocumented changes

DEMO

Much more!
Chef-server
Searching
Tagging
Libraries

Resources
OpsCode: http://www.opscode.com/
Chef Wiki: http://wiki.opscode.com/
37signals recipes:
http://github.com/37signals/37s_cookbooks
Engine Yard recipes:
http://github.com/engineyard/ey-cloud-recipes

Me!
Twitter: @krobertson
Blog: http://invalidlogic.com/
Email: ken@invalidlogic.com

Questions?