Chef introduction and overview of Chef for OpenStack

1. Chef for OpenStack
Matt Ray
matt@opscode.com
IRC/Twitter/GitHub: mattray
www.opscode.com/openstack

2. Deploying and Managing
OpenStack is not simple.

3. Chef makes it easier.

4. See Node
Application Server

5. See Nodes
Application Server
Application Database

6. See Nodes Grow
Application Server
Application Databases

7. See Nodes Grow
Application Servers
Application Databases

8. See Nodes Grow
Load Balancer
Application Servers
Application Databases

9. See Nodes Grow
Load Balancers
Application Servers
Application Databases

10. See Nodes Grow
Load Balancers
Application Servers
Application Database Cache
Application Databases

11. Tied together with Config
Load Balancers
Application Servers
Application Database Cache
Application Databases

12. Infrastructure is a Snowflake
Load Balancers
Application Servers
Application Database Cache
Floating IP?
Application Databases

13. Evolving Complexity
Load Balancers
ApplicationCache
Application Servers
NoSQL Database Cache
Database Slaves
Database

14. Complexity Grows Quickly
DC2
DC1
DC3

15. And it Continues to Evolve
Tell me about
Chef!
http://www.flickr.com/photos/16339684@N00/2681435235/

16. Chef is Infrastructure as Code
• Programmatically
provision and configure
• Treat like any other code
base
• Reconstruct business from
code repository, data
backup, and bare metal
resources.
http://www.flickr.com/photos/louisb/4555295187/

17. Nodes
• Chef-Client generates
configurations directly
on nodes from their
run list
• Reduce management
complexity through
abstraction
• Store the configuration
of your programs in
version control
http://www.flickr.com/photos/ssoosay/5126146763/

18. Collections of Resources
• Networking • Routes
• Users
• Files • Groups
• Directories • Tasks
• Symlinks • Packages
• Mounts • Software
• Services
• Configurations
• Other Stuff
http://www.flickr.com/photos/stevekeys/3123167585/

19. Declarative Interface to Resources
• Define policy
• Say what, not how
• Pull not Push
http://www.flickr.com/photos/bixentro/2591838509/

20. Ruby!
extra_packages = case node['platform']
when "ubuntu","debian"
%w{
ruby1.8
ruby1.8-dev
rdoc1.8
ri1.8
libopenssl-ruby
}
end
extra_packages.each do |pkg|
package pkg do
action :install
end
end

21. Recipes and Cookbooks
• Recipes are collections of
Resources
• Cookbooks contain
recipes, templates, files,
custom resources, etc
• Code re-use and
modularity
• Hundreds already on
Community.opscode.com
http://www.flickr.com/photos/shutterhacks/4474421855/

22. Search
• Search for nodes
with Roles
• Find configuration
data
• IP addresses
• Hostnames
• FQDNs
http://www.flickr.com/photos/kathycsus/2686772625

23. Pass Results to Templates
pool_members = search("node","role:webserver”)
template "/etc/haproxy/haproxy.cfg" do
source "haproxy-app_lb.cfg.erb"
owner "root"
group "root"
mode 0644
variables :pool_members => pool_members.uniq
notifies :restart, "service[haproxy]"
end

24. Pass Results to Templates
# Set up application listeners here.
listen application 0.0.0.0:80
balance roundrobin
<% @pool_members.each do |member| -%>
server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check
<% end -%>
<% if node["haproxy"]["enable_admin"] -%>
listen admin 0.0.0.0:22002
mode http
stats uri /
<% end -%>

25. So when this
Graphite Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master

26. Becomes this
Graphite Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master

27. Updates can be automatic
Graphite Nagios
Jboss App
Memcache
Postgres Slaves
Postgres Master

28. Count the resources
• Load balancer config
Graphite Nagios • Nagios host ping
• Nagios host ssh
Jboss App • Nagios host HTTP
• Nagios host app health
Memcache • Graphite CPU
• Graphite Memory
Postgres Slaves • Graphite Disk
• Graphite SNMP
• Memcache firewall
• 12+ resource changes for 1 node addition • Postgres firewall
Postgres authZ config

29. Build anything
• Simple internal applications
• Complex external applications
• Workstations
• Hadoop clusters
• IaaS infrastructure
• PaaS infrastructure
• SaaS applications
• Storage systems
• You name it
http://www.flickr.com/photos/hyku/245010680/

30. And manage it simply
• Automatically
reconfigure
everything
• Linux, Windows,
Unixes, BSDs
• Load balancers
• Metrics collection
systems
• Monitoring systems
• Cloud migrations
become trivial
http://www.flickr.com/photos/helico/404640681/

31. The Chef Community
• Apache License, Version 2.0
• 800+ Individual contributors
• 150+ Corporate contributors
• HP, Dell, Rackspace, VMware, Joyent,
Calxeda, Heroku, SUSE and many more
• 500+ cookbooks
• http://community.opscode.com

32. What's Out There?
Chef for OpenStack
Resources

33. Chef for OpenStack: Why
• Community for the automated deployment
and management of OpenStack
• Reduce fragmentation and encourage
collaboration
• Deploying OpenStack is not "secret sauce"
• Project not a product
• Apache 2 license

34. Chef for OpenStack: What
• Chef Repository for Deploying OpenStack
• Cookbooks
• Keystone
• Glance
• Nova
• Horizon
• Swift
• Knife OpenStack

35. Chef for OpenStack: Where
• opscode.com/openstack
• groups.google.com/group/opscode-chef-
openstack
• #openstack-chef on irc.freenode.net
• github.com/opscode/openstack-chef-repo
• github.com/opscode-cookbooks/
• keystone, glance, nova, horizon, swift
• github.com/opscode/knife-openstack

36. Chef for OpenStack: Who

37. Chef for OpenStack: When
• Essex is working
• KVM
• Ubuntu 12.04
• more to come!
• Folsom about to start
• Milestones for features
• Grizzly and forward

38. Deploying OpenStack
• Chef ties it all together automatically
• Scaling changes how we deploy
• Interchangeable components
• Configurations shared, supported &
documented
• Licensing makes it available to everyone

39. Rackspace Cloud Builders
• Source of current cookbooks
• github.com/rcbops/chef-cookbooks
• www.rackspace.com/cloud/private/

40. knife openstack

41. knife openstack
$ knife openstack
Available openstack subcommands: (for details, knife SUB-
COMMAND --help)
** OPENSTACK COMMANDS **
knife openstack flavor list (options)
knife openstack image list (options)
knife openstack server create (options)
knife openstack server delete SERVER [SERVER] (options)
knife openstack server list (options)

42. knife openstack flavor list
$ knife openstack flavor list
ID Name Virtual CPUs RAM Disk
1 m1.tiny 1 512 MB 0 GB
2 m1.small 1 2048 MB 20 GB
3 m1.medium 2 4096 MB 40 GB
4 m1.large 4 8192 MB 80 GB
5 m1.xlarge 8 16384 MB 160 GB

43. knife openstack image list
$ knife openstack image list
ID Name
13 natty-server-cloudimg-amd64
12 natty-server-cloudimg-amd64-kernel
15 oneiric-server-cloudimg-amd64
14 oneiric-server-cloudimg-amd64-kernel

44. knife openstack server create
knife openstack server create --node-name ko1 --flavor 1 --image 13 -S trystack

46. $ ssh -i ~/.ssh/trystack.pem ubuntu@8.21.28.24
The authenticity of host '8.21.28.24 (8.21.28.24)' can't be established.
RSA key fingerprint is 0c:d8:3e:34:d1:de:c4:ee:5f:bc:b5:89:11:0d:73:e0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '8.21.28.24' (RSA) to the list of known hosts.
Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-13-virtual x86_64)
* Documentation: https://help.ubuntu.com/
System information as of Thu Feb 16 23:43:29 UTC 2012
System load: 0.08 Processes: 63
Usage of /: 40.8% of 1.35GB Users logged in: 0
Memory usage: 6% IP address for eth0: 8.21.28.24
Swap usage: 0%
---------------------------------------------------------------------
<snip>
Get cloud support with Ubuntu Advantage Cloud Guest
http://www.ubuntu.com/business/services/cloud
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@ko1:~$

47. Chef for Infrastructure Portability
• knife ec2
• knife rackspace
• knife hp
• knife openstack
• ... and many others

48. Chef for OpenStack Roadmap
• Documentation
• Hypervisors (LXC, Xen, Hyper-V)
• Databases (PostgreSQL)
• Operating Systems (RHEL)
• HA Configuration
• Quantum
• Cinder

49. Chef for OpenStack Ecosystem
• Cookbooks reusable outside of OpenStack
• Community Events
• Librarian
• Spiceweasel
• pxe_dust
• Crowbar

50. Thanks!
Matt Ray
matt@opscode.com
IRC/Twitter/GitHub: mattray
www.opscode.com/openstack