5. Infrastructure
"It is common to think in terms of individual machines
rather than view an entire infrastructure as a
combined whole"
âA good infrastructure, whether departmental,
divisional, or enterprise-wide, is a single loosely-
coupled virtual machine, with hundreds or
thousands of hard drives and CPU's.â
-- Bootstrapping an Infrastructure USENIX LISA â98
http://www.infrastructures.org/papers/bootstrap/bootstrap.html
6. .... as code!
⢠Programmatically provision and
configure
⢠Treat like any other code base
⢠Reconstruct operations from code
repository, data backup, and bare
metal resources.
http://www.ďŹickr.com/photos/louisb/4555295187/
7. Considerations
⢠Infrastructure changes over time
⢠Entropy
⢠Changing business requirements
http://www.ďŹickr.com/photos/seatbelt67/502255276/
10. Manual
ConďŹguration
⢠Labor intensive
⢠Error prone
⢠Hard to reproduce
⢠Unsustainable
http://www.ďŹickr.com/photos/pureimaginations/4805330106/
11. Scripting
⢠Typically very brittle
⢠Throw away, one off scripts
⢠grep sed awk perl
⢠curl | bash
http://www.ďŹickr.com/photos/40389360@N00/2428706650/
12. File
Distribution
⢠NFS mounts
⢠rdist
⢠scp-on-a-for-loop
⢠rsync on cron
http://www.ďŹickr.com/photos/walkadog/4317655660
13. This used to be
awesome
for i in `cat servers.txt` ; do scp ntp.conf root@$i:/etc/
ntpd.conf ; done
for i in `cat servers.txt` ; do ssh root@$i /etc/init.d/ntpd
restart ; done
for i in `cat servers.txt` ; do ssh root@$i chkconfig ntpd
on ; done
⢠^ does not scale
http://www.ďŹickr.com/photos/alexerde/3479006495
14. Declarative
Syntax
⢠Define policy
⢠Say what, not how
⢠Abstract interface to
resources
⢠Enables some interesting
behavior
http://www.ďŹickr.com/photos/bixentro/2591838509/
16. Declarative Syntax
package "ntp" do
action :install
Idempotence
â˘
end
cookbook_ďŹle "/etc/ntp.conf" do
Youâll hear this a lot
source "ntp.conf"
owner "root" ⢠Property of declarative
group "root" interface
mode 0644
action :create
notiďŹes :restart, âservice[ntpd]â
⢠Eliminates brittleness of
end
scripting
service "ntpd" do
action [:enable,:start]
⢠Identity function: f(x)=x
end
17. Declarative Syntax
while true do
package "ntp" do
action :install
Idempotence
â˘
end
cookbook_ďŹle "/etc/ntp.conf" do
Youâll hear this a lot
source "ntp.conf"
owner "root" ⢠Property of declarative
group "root" interface
mode 0644
action :create
notiďŹes :restart, âservice[ntpd]â
⢠Eliminates brittleness of
end
scripting
service "ntpd" do
action [:enable,:start]
⢠Identity function: f(x)=x
end ⢠Safe to repeat
end
18. Declarative Syntax
Convergence
⢠Agents âconvergeâ a system to
desired state
⢠Repetition inches closer to
desired state
⢠It eventually gets there
⢠SCIENCE!
http://www.ďŹickr.com/photos/tolomea/4852616645/
19. Declarative Syntax
Convergence
service "ntpd" do
action [:enable,:start]
ignore_failure true
end
cookbook_ďŹle "/etc/ntp.conf" do ⢠Agents âconvergeâ a system to
source "ntp.conf" desired state
owner "root"
group "root"
mode 0644 ⢠Repetition inches closer to
action :create desired state
â˘
notiďŹes :restart, âservice[ntpd]â
ignore_failure true It eventually gets there
â˘
end
SCIENCE!
package "ntp" do
action :install
ignore_failure true
end
20. Declarative Syntax
# echo âboomâ > /etc/ntp.conf ;
chef-client
Convergence
$ grep server /etc/ntp.conf | head -n 1
us.pool.ntp.org
$ ps -e | grep ntp
⢠Fights entropy, unauthorized
1799 ? 00:00:00 ntpd
changes, and gingivitis
# /etc/init.d/ntpd stop ; chef-client ⢠Update function inputs to deal
with changing requirements
ps -e | grep ntp
1822 ? 00:00:00 ntpd
21. ConďŹg Generation
⢠Often made by hand (still!?)
⢠Stop that.
⢠Generate them based on
database content
⢠Infrastructures evolve
http://www.flickr.com/photos/jabella/4753170413/
33. Generate conďŹgs
⢠Centralized generation
⢠Version control!
⢠Distribute with packages, Chef,
git, whatever.
http://www.flickr.com/photos/ssoosay/5126146763/
34. Generate conďŹgs
⢠Local generation directly on nodes
⢠Reduces management complexity
⢠No need to distribute
⢠Version control the programs instead
http://www.flickr.com/photos/ssoosay/5126146763/
36. All That Stuff
⢠Declarative interface to resources
⢠Database of nodes and their roles
⢠Grab remote configs
⢠Generate configs locally
37. and more!
⢠Data Driven Infrastructure
⢠Use APIs to obtain data
⢠chef-server, SQL, anything.
⢠Feed resources parameters
⢠IPs, FQDNs, memory sizes,
⢠Templates, package, firewall
rules
38. Architecture
⢠Code Repository
⢠Chef Server
⢠Chef Clients
⢠Data Bags
⢠Recipes and Cookbooks
⢠Roles and Run Lists
http://www.flickr.com/photos/boedker/3871267007
39. Code Repository
⢠Version control
⢠Development
workflows
⢠Sharing is Caring
40. Chef Server
Server
Server
chef-server
Server
Server
⢠Upload from laptop
with knife
RESTful API
Cookbook
Cookbook
Cookbook
Data Bag
Knife Knife
Role
Knife
41. Chef Clients
Server
Server
chef-server
Server
Server
Knife
⢠Clients are API users
⢠Read RESTful API Knife
⢠Write
⢠Search
chef-client
chef-client chef-client chef-client chef-client
42. Chef Clients
Server
Server
someara.pub chef-server
Server
Server
jtimberman.pub
node5.fqdn.pub
Knife someara.pem
⢠Clients are API users
⢠Public keys on server RESTful API Knife jtimberman.pem
⢠Private keys local to
machines
chef-client
chef-client chef-client chef-client chef-client
node5.fqdn.pem
43. Run Lists
Server
Server
chef-server
Server
Server
Ohai!
API chef-client
Give me
recipe[ntp::client]
ntp
node
client.rb
44. Run Lists
Server
Server
chef-server
Server
Server
Ohai!
chef-client
API Give me
ântp::clientâ,
ntp âopenssh::serverâ
openssh
node
client.rb
server.rb
45. Run Lists
Server
Server
chef-server
Server
Server
Ohai!
chef-client Give me
API
ârecipe[ntp::client]â,
ntp ârecipe[openssh::server]â,
ârecipe[apache]â,
openssh
node ârecipe[php]â
client.rb apache
server.rb php
default.rb
default.rb
46. Roles
Server
Server
chef-server
Server
Server
Role Recipe
API
Role
Role Recipe
Role Recipe
Recipe
Knife
Recipe
Recipe
Recipe
47. Roles
Server
Server
chef-server
Server
Server
chef-client Ohai!
API
Give me
ntp ârole[base]â,
ârole[webserver]â
openssh
node
client.rb apache
server.rb php
default.rb
default.rb
48. Roles
Server
Server
chef-server
Server
Server
ntp
openssh
chef-client
API client.rb apache
php
server.rb ârole[webserver]â
default.rb
ntp default.rb node
openssh
chef-client
client.rb mysql
server.rb
server.rb
ârole[database]â
node
49. Bootstrapping
nodes
⢠Get chef-client installed
⢠Write run list to a file
⢠âPress goâ
http://www.flickr.com/photos/liftarn/1447521121/
51. Bootstrapping nodes
{
"kernel": {
"machine": "x86_64",
"name": "Darwin",
"os": "Darwin",
"version": "Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010;
â˘
root:xnu-1504.7.4~1/RELEASE_I386",
Ohai generates a JSON },
"release": "10.4.0"
attributes list "platform_version": "10.6.4",
â˘
"platform": "mac_os_x",
Run list and attributes are "platform_build": "10F569",
"domain": "local",
combined into a Node object "os": "darwin",
"current_user": "mray",
â˘
"ohai_time": 1278602661.60043,
Can be viewed and "os_version": "10.4.0",
"uptime": "18 days 17 hours 49 minutes 18 seconds",
searched through API "ipaddress": "10.13.37.116",
"hostname": "morbo",
"fqdn": "morbomorbo.local",
"uptime_seconds": 1619358
}
52. Bootstrapping nodes
⢠Run list is requested
⢠Cookbooks downloaded
⢠Recipes executed
⢠Node saved to chef-server
http://www.flickr.com/photos/architopher/457885721
53. Cookbooks
and Recipes
⢠Cookbooks contain recipes
⢠And everything they need to
work
⢠Templates, files, custom
resources, etc
http://www.flickr.com/photos/shutterhacks/4474421855/
54. Cookbooks
$ tree -a cookbooks/haproxy/
README.md
attributes
  default.rb
⢠Cookbooks contain recipes metadata.rb
⢠And everything they need to recipes
work   app_lb.rb
  default.rb
⢠Templates, files, custom templates
resources, etc default
haproxy-app_lb.cfg.erb
haproxy-default.erb
haproxy.cfg.erb
55. Recipes
package "haproxy" do
action :install
end
template "/etc/default/haproxy" do
source "haproxy-default.erb"
⢠Recipes contain lists of owner "root"
group "root"
resources mode 0644
notiďŹes :restart, "service[haproxy]"
end
service "haproxy" do
action [:enable, :start]
end
57. Resources
package "apache2" do
version "2.2.11-2ubuntu2.6"
action :install
end
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
action :create
end
58. Resources
package "apache2" do
â˘
version "2.2.11-2ubuntu2.6"
Have a type action :install
end
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
action :create
end
59. Resources
package "apache2" do
â˘
version "2.2.11-2ubuntu2.6"
Have a type action :install
â˘
end
Have a name
template "/etc/apache2/apache2.conf" do
source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
action :create
end
60. Resources
package "apache2" do
â˘
version "2.2.11-2ubuntu2.6"
Have a type action :install
â˘
end
Have a name
â˘
template "/etc/apache2/apache2.conf" do
Have parameters source "apache2.conf.erb"
owner "root"
group "root"
mode 0644
action :create
end
61. Resources
package "apache2" do
â˘
version "2.2.11-2ubuntu2.6"
Have a type action :install
â˘
end
Have a name
â˘
template "/etc/apache2/apache2.conf" do
Have parameters source "apache2.conf.erb"
owner "root"
⢠Take action to put the resource group "root"
mode 0644
in the declared state action :create
end
63. Searching
⢠All object in Chef server are
indexed by Solr
http://www.flickr.com/photos/fotos_medem/3399096196/
64. Searching
⢠All object in Chef server are
indexed by Solr
⢠Can search through the API
http://www.flickr.com/photos/fotos_medem/3399096196/
65. Searching
⢠All object in Chef server are
indexed by Solr
⢠Can search through the API
⢠From knife and in recipes
http://www.flickr.com/photos/fotos_medem/3399096196/
66. Searching
⢠All object in Chef server are
indexed by Solr
⢠Can search through the API
⢠From knife and in recipes
⢠Returns an array of JSON Node
objects
http://www.flickr.com/photos/fotos_medem/3399096196/