SlideShare uma empresa Scribd logo
1 de 193
Baixar para ler offline
Chef Fundamentals by Chef Software, Inc. is licensed under a
Creative Commons Attribution-ShareAlike 4.0 International License.
Intro to Infrastructure as Code
Introduction to Chef
POSSCON – April 2015
http://bit.ly/posscon2015-chef
Nathen Harvey
• Community Director at Chef
• Co-host of the Food Fight Show
• Co-organizer of DevOpsDC meetup
• Occasional farmer – http://bit.ly/farmer-nathen
• Love Eggs – http://eggs.chef.io
• @nathenharvey
• nharvey@chef.io
Hello!
• System Administrator?
Hello!
• System Administrator?
• Developer?
Hello!
• System Administrator?
• Developer?
• Ruby developer?
Hello!
• System Administrator?
• Developer?
• Ruby developer?
• DevOp?
Hello!
• System Administrator?
• Developer?
• Ruby developer?
• DevOp?
• Business Person?
Are you experienced?
• Experience with Infrastructure as Code or
Configuration Management?
Are you experienced?
• Experience with Infrastructure as Code or
Configuration Management?
• Experience with Chef?
Which version control system do your use?
Which version control system do your use?
• cp foo foo.bak
Which version control system do your use?
• cp foo foo.bak
• cp foo{,.`date +%Y%m%d%H%M`}
Which version control system do your use?
• cp foo foo.bak
• cp foo{,.`date +%Y%m%d%H%M`}
• cp foo{,.`date +%Y%m%d%H%M`-`$USER`}
Infrastructure as Code
The Sys Admin’s Journey
• ssh
The Sys Admin’s Journey
• ssh
• Store notes in ~/server.txt
The Sys Admin’s Journey
• ssh
• Store notes in ~/server.txt
• Move notes to the wiki
The Sys Admin’s Journey
• ssh
• Store notes in ~/server.txt
• Move notes to the wiki
• Write some scripts (setup.sh, fixit.sh, etc.)
The Sys Admin’s Journey
• ssh
• Store notes in ~/server.txt
• Move notes to the wiki
• Write some scripts (setup.sh, fixit.sh, etc.)
• Golden images and snapshots
The Sys Admin’s Journey
• ssh
• Store notes in ~/server.txt
• Move notes to the wiki
• Write some scripts (setup.sh, fixit.sh, etc.)
• Golden images and snapshots
• Remote execution via ssh
The Sys Admin’s Journey
• ssh
• Store notes in ~/server.txt
• Move notes to the wiki
• Write some scripts (setup.sh, fixit.sh, etc.)
• Golden images and snapshots
• Remote execution via ssh
• Policy-driven configuration management
Benefits of Automation
Dimensions of Scale
Automation Platform
• Creates a dependable view of your entire network’s
state.
• Can handle complex dependencies among the
nodes of your network.
• Is fault tolerant.
• Is secure.
• Can handle multiple platforms
• Can manage cloud resources
• Provides a foundation for innovation
Infrastructure as Code
• Programmatically
provision and configure
components
Infrastructure as Code
• Treat like any other
code base
Infrastructure as Code
• Reconstruct business
from code repository,
data backup, and
compute resources
Infrastructure as Code
• Programmatically
provision and configure
components
• Treat like any other
code base
• Reconstruct business
from code repository,
data backup, and
compute resources
Policy-based
• You capture the policy for your infrastructure in
code
• A program ensures each node in your
infrastructure complies with the policy
• A control loop keeps the system stable and
allows for change when policy is updated
Sample Infrastructure
Graphite Nagios
Rails
Memcache
Postgres Slaves
Postgres Master
New Compliance Mandate!
Graphite Nagios
Rails
Memcache
Postgres Slaves
Postgres Master
New Compliance Mandate!
Graphite Nagios
Rails
Memcache
Postgres Slaves
Postgres Master
•  Move SSH off of port 22
•  Let’s put it on 2022
6 Golden Images to Update
Graphite Nagios
Rails
Memcache
Postgres Slaves
Postgres Master
1
3
4
5
6
2
/etc/ssh/sshd_config
--- a/sshd_config
+++ b/sshd_config
-Port 22
+Port 2202
12 Instances to replace
Graphite Nagios
Rails
Memcache
Postgres Slaves
Postgres Master
1
3
2
5 6 74
8
10
9
11
12
•  Launch
•  Delete
•  Repeat
•  Typically manually
Done in maintenance window
Graphite Nagios
Rails
Memcache
Postgres Slaves
Postgres Master
1
3
2
5 6 74
8
10
9
11
12
•  High stakes
•  Late hours
•  Risky change
New configurations required?
Graphite Nagios
Rails
Memcache
Postgres Slaves
Postgres Master
Do the new instances
have new IP Addresses?
* Not all connections shown
Chef
Fast, scalable, flexible IT automation
What is Chef
• Open source framework for managing
complexity in your infrastructure through policy-
driven automation code
• A community of professionals
• A company
Chef
https://www.chef.io/chef/
Chef Server – Policy & State
Desired Configuration
Node
Chef
Server
chef-client
What policy should I follow?
Desired Configuration
Node
Chef
Server
chef-client
What policy should I follow?
"recipe[ntp::client]"
"recipe[users]"
"role[webserver]"
Desired Configuration
Chef
Server
chef-client
What policy should I follow?
"recipe[ntp::client]"
"recipe[users]"
"role[webserver]"
Chef Server – Policy & State
HA Proxy Configuration
Webservers
HA Proxy
HA Proxy Configuration
Webservers
HA ProxyChef
Server
HA Proxy Configuration
Webservers
HA ProxyChef
Server
Webservers?
HA Proxy Configuration
Webservers
HA ProxyChef
Server
Webservers?
HA Proxy Configuration
Webservers
HA ProxyChef
Server
Webservers?
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
HA Proxy Configuration
Webservers
HA ProxyChef
Server
Webservers?
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
HA Proxy Configuration
Webservers
HA ProxyChef
Server
Webservers?
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
pool_members
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
HA Proxy Configuration
Webservers
HA Proxy
haproxy.cfg
server web01 10.1.1.1 weight 1 maxconn 1 check
server web02 10.1.1.2 weight 1 maxconn 1 check
server web03 10.1.1.3 weight 1 maxconn 1 check
server web04 10.1.1.4 weight 1 maxconn 1 check
pool_members
{
"web01" : {
"hostname" : "web01",
"ipaddress" : "10.1.1.1"
},
"web02" : {
"hostname" : "web02",
"ipaddress" : "10.1.1.2"
},
"web03" : {
"hostname" : "web03",
"ipaddress" : "10.1.1.3"
},
"web04" : {
"hostname" : "web04",
"ipaddress" : "10.1.1.4"
}
HA Proxy Configuration
Webservers
HA Proxy
Building your policy
Resources and Recipes
Resources
• Piece of the system and its desired state
Resources - Package
Package that should be
installed
package "mysql-server" do
action :install
end
Resources - Service
Service that should be
running and restarted
on reboot
service "iptables" do
action [ :start, :enable ]
end
Resources - Service
File that should be
generated
file "/etc/motd" do
content "Property of Chef Software"
end
Resources - Cron
Cron job that should be
configured
cron "restart webserver" do
hour '2'
minute '0'
command 'service httpd restart'
end
Resources - User
User that should be
managed
user "nginx" do
comment "Nginx user <nginx@example.com>"
uid 500
gid 500
supports :manage_home => true
end
Resources - DSC
DSC resource that
should be run
dsc_script 'emacs' do
code <<-EOH
Environment 'texteditor'
{
Name = 'EDITOR'
Value = 'c:emacsbinemacs.exe'
}
EOH
end
Resources – Registry Key
Registry key that should
be created
registry_key "HKEY_LOCAL_MACHINE
SOFTWAREMicrosoftWindows
CurrentVersionPoliciesSystem"
do
values [{
:name => "EnableLUA",
:type => :dword,
:data => 0
}]
action :create
end
Resources
• Piece of the system and its desired state
• http://docs.chef.io/chef/resources.html
Lab – Install a text editor
• Problem: Our workstation does not have
$EDITOR installed
• Success Criteria: You can edit files with
$EDITOR
• $EDITOR is your favorite command line text
editor: vim, emacs, or nano
What’s up with the card?
• http://bit.ly/posscon-workstations
• Login: chef
• Password: [REDACTED]
$
The authenticity of host '54.165.227.226
(54.165.227.226)' can't be established.
RSA key fingerprint is c1:ec:ab:66:fb:22:4a:
8f:c2:c5:9b:26:77:f3:dd:b3.
Are you sure you want to continue connecting
(yes/no)? yes
Warning: Permanently added
'54.165.227.226' (RSA) to the list of known
hosts.
chef@54.165.227.226's password:
Login to your lab machine
ssh chef@54.164.75.30
Welcome to your workstation
• ChefDK version 0.4.0 is installed
• chef --version
• Chef user has passwordless sudo access
• sudo cat /etc/shadow
$
/usr/bin/which: no vim in (/opt/
chefdk/bin:/home/chef/.chefdk/gem/
ruby/2.1.0/bin:/opt/chefdk/embedded/
bin:/usr/local/bin:/bin:/usr/bin:/
usr/local/sbin:/usr/sbin:/sbin:/
home/chef/bin)
Is $EDITOR installed?
which vim
chef-apply
• chef-apply is an executable program that allows
you to work with resources
• Is included as part of the ChefDK
• A great way to explore resources
• NOT how you’ll eventually use Chef in
production
$
Recipe: (chef-apply cookbook)::(chef-apply recipe)
* package[vim] action install
- install version 7.2.411-1.8.el6 of package vim-enhanced
Install vim
sudo chef-apply -e "package 'vim'"
$
Recipe: (chef-apply cookbook)::(chef-apply recipe)
* package[emacs] action install
- install version 23.1-25.el6 of package emacs
Install emacs
sudo chef-apply -e "package 'emacs'"
$
Recipe: (chef-apply cookbook)::(chef-apply
recipe)
* package[nano] action install
- install version 2.0.9-7.el6 of package nano
Install nano
sudo chef-apply -e "package 'nano'"
Resources
• Describe the desired state
• Do not need to tell Chef how to get there
• What happens if you re-run the chef-apply
command?
$
Recipe: (chef-apply cookbook)::(chef-apply
recipe)
* package[vim] action install (up to date)
Install $EDITOR again with chef-apply
sudo chef-apply -e "package 'vim'"
Test and Repair
Resources follow a test
and repair model
package "vim"
Test and Repair
Resources follow a test
and repair model
package "vim"
Is vim installed?
Test and Repair
Resources follow a test
and repair model
package "vim"
Is vim installed?
Yes
Test and Repair
Resources follow a test
and repair model
package "vim"
Is vim installed?
Done
Yes
Test and Repair
Resources follow a test
and repair model
package "vim"
Is vim installed?
Done
Yes No
Test and Repair
Resources follow a test
and repair model
package "vim"
Is vim installed?
Done Install it
Yes No
Test and Repair
Resources follow a test
and repair model
package "vim"
Is vim installed?
Done Install it
Yes No
Resources – Test and Repair
• Resources follow a test and repair model
• Resource currently in the desired state? (test)
• Yes – Do nothing
• No – Bring the resource into the desired state
(repair)
Resources
• package
• template
• service
• directory
• user
• group
• dsc_script
• registry_key
• powershell_script
• cron
• mount
• route
• …and more!
OPEN IN EDITOR:
SAVE FILE!
Hello, Chef!
file "/tmp/hello_chef.txt" do
content "Hello, Chef"
mode "0777"
end
~/hello_chef.rb
$
Recipe: (chef-apply cookbook)::(chef-apply recipe)
* file[/tmp/hello_chef.txt] action create
- create new file /tmp/hello_chef.txt
- update content in file /tmp/hello_chef.txt from none to 79c290
--- /tmp/hello_chef.txt 2014-10-22 19:59:04.000000000 -0400
+++ /tmp/.hello_chef.txt20141022-23075-19aelx1 2014-10-22
19:59:04.000000000 -0400
@@ -1 +1,2 @@
+Hello, Chef
- change mode from '' to '0777'
Apply the policy
sudo chef-apply hello_chef.rb
Resources
• Describe the desired state
• Do not need to tell Chef how to get there
• What happens when you re-apply the policy?
$
Recipe: (chef-apply cookbook)::(chef-apply recipe)
* file[/tmp/hello_chef.txt] action create (up to date)
Apply the policy
sudo chef-apply hello_chef.rb
Resources
• A piece of the system
• Its desired state
file "/tmp/hello_chef.txt" do
content "Hello, Chef"
mode "0777"
end
$
Change the state of the system
echo “Hello, #posscon” > /tmp/hello_chef.txt
$
Recipe: (chef-apply cookbook)::(chef-apply recipe)
* file[/tmp/hello_chef.txt] action create
- update content in file /tmp/hello_chef.txt from e453df to 79c290
--- /tmp/hello_chef.txt 2014-10-22 20:00:20.000000000 -0400
+++ /tmp/.hello_chef.txt20141022-23340-17a7m5t 2014-10-22
20:00:50.000000000 -0400
@@ -1,2 +1,2 @@
-“Hello, #posscon”
+Hello, Chef
Apply the policy
sudo chef-apply hello_chef.rb
Resources – Test and Repair
• Resources use a test and repair model
• Resource currently in the desired state?
• Yes – Do nothing
• No – Bring the resource into the desired state (repair)
Built-in Resources
• package
• template
• service
• cron
• directory
• mount
• user
• group
• registry_key
• remote_directory
• route
• and many more…
docs.chef.io/chef/resources.html
Recipes
• Policy is defined as a collection of resources in
recipes. There are lots of abstractions on top of
this but resources are the basic building blocks.
Install git
• The package git should be installed
• The file named '/home/chef/.gitconfig'
should be created.
• It should be owned by the chef user and group.
• It should have the content:
[user]n name=John Doen email=jdoe@examplen
OPEN IN EDITOR:
SAVE FILE!
Install git
package 'git' do
action :install
end
file '/home/chef/.gitconfig' do
content "[user]n name=John Doen email=jdoe@examplen"
user 'chef'
group 'chef'
end
~/git.rb
$
Recipe: (chef-apply cookbook)::(chef-apply recipe)
* package[git] action install
- install version 1.7.1-3.el6_4.1 of package git
* file[/home/chef/.gitconfig] action create
- create new file /home/chef/.gitconfig
- update content in file /home/chef/.gitconfig from none to 259950
--- /home/chef/.gitconfig 2014-09-24 00:24:13.558127555 +0000
+++ /tmp/..gitconfig20140924-10180-1ij68vq 2014-09-24 00:24:13.559127555 +0000
@@ -1 +1,4 @@
+[user]
+ name=John Doe
+ email=jdoe@example.com
- change owner from '' to 'chef'
- change group from '' to 'chef'
- restore selinux security context
Install git
sudo chef-apply ~/git.rb
Test-driven Infrastructure
Change policy with confidence
Our process
• Write policy
• Apply policy
• Verify policy
• Not bad for the simple case, will quickly get
untenable
Faster Feedback
• Speed-up the feedback loops with automated
testing.
• Have confidence in your changes before you run
them in production
Chef Testing
• Did chef-client complete successfully?
• Did the recipe put the node in the desired state?
• Are the resources properly defined?
• Does the code follow our style guide?
Test-driving infrastructure
• We are going to use a relatively simple scenario
• We are going to explore many facets of testing
• We are going to follow a test-first, test-driven
model
Our Scenario
• We want a custom home page available on the
web.
$
Compiling Cookbooks...
Recipe: code_generator::cookbook
* directory[/home/chef/chef-repo/cookbooks/apache] action create
- create new directory /home/chef/chef-repo/cookbooks/apache
* template[/home/chef/chef-repo/cookbooks/apache/metadata.rb] action
create_if_missing
- create new file /home/chef/chef-repo/cookbooks/apache/metadata.rb
- update content in file /home/chef/chef-repo/cookbooks/apache/
metadata.rb from none to 4c0e2d
(diff output suppressed by config)
* template[/home/chef/chef-repo/cookbooks/apache/README.md] action
create_if_missing
Create an apache cookbook
chef generate cookbook cookbooks/apache
Questions to ask when testing
• Did chef-client complete successfully?
• Did the recipe put the node in the desired state?
• Are the resources properly defined?
• Does the code following our style guide?
Chef client success status
• Requirements to verify chef-client success:
• A place to store the cookbook artifact
Chef client success status
• Requirements to verify chef-client success:
• A place to store the cookbook artifact
• A chef-client with access to the cookbook
Chef client success status
• Requirements to verify chef-client success:
• A place to store the cookbook artifact
• A chef-client with access to the cookbook
• A target server running the same OS as production
Test Kitchen
• Test harness to execute code on one
or more platforms
• Driver plugins to allow your code to
run on various cloud and virtualization
providers
• Includes support for many testing
frameworks
• Included with ChefDK
Test Matrix
• Two operating systems ubuntu-12.04
centos-6.4
Test Matrix
• Two operating systems
• One recipe
default
ubuntu-12.04 apache::default
centos-6.4 apache::default
Test Matrix
• Two operating systems
• Two recipes
default ssl
ubuntu-12.04 apache::default apache::ssl
centos-6.4 apache::default apache::ssl
Test Matrix
• Three operating
systems
• Two recipes
default ssl
ubuntu-12.04 apache::default apache::ssl
centos-6.4 apache::default apache::ssl
ubuntu-14.04 apache::default apache::ssl
OPEN IN EDITOR:
SAVE FILE!
Configuring the Kitchen
---
driver:
name: vagrant
provisioner:
name: chef_zero
platforms:
- name: ubuntu-12.04
- name: centos-6.4
suites:
- name: default
run_list:
- recipe[apache::default]
attributes:
apache/.kitchen.yml
.kitchen.yml
• driver - virtualization or
cloud provider
---
driver:
name: vagrant
provisioner:
name: chef_zero
platforms:
- name: ubuntu-12.04
- name: centos-6.4
suites:
- name: default
run_list:
- recipe[apache::default]
attributes:
.kitchen.yml
• provisioner -
application to configure
the node
---
driver:
name: vagrant
provisioner:
name: chef_zero
platforms:
- name: ubuntu-12.04
- name: centos-6.4
suites:
- name: default
run_list:
- recipe[apache::default]
attributes:
.kitchen.yml
• platforms - target
operating systems
---
driver:
name: vagrant
provisioner:
name: chef_zero
platforms:
- name: ubuntu-12.04
- name: centos-6.4
suites:
- name: default
run_list:
- recipe[apache::default]
attributes:
.kitchen.yml
• suites - target
configurations
---
driver:
name: vagrant
provisioner:
name: chef_zero
platforms:
- name: ubuntu-12.04
- name: centos-6.4
suites:
- name: default
run_list:
- recipe[apache::default]
attributes:
.kitchen.yml
---
driver:
name: vagrant
provisioner:
name: chef_zero
platforms:
- name: ubuntu-12.04
- name: centos-6.4
suites:
- name: default
run_list:
- recipe[apache::default]
default
ubuntu-12.04 apache::default
centos-6.4 apache::default
.kitchen.yml
---
driver:
name: vagrant
provisioner:
name: chef_zero
platforms:
- name: ubuntu-12.04
- name: centos-6.4
suites:
- name: default
run_list:
- recipe[apache::default]
- name: ssl
run_list:
- recipe[apache::ssl]
default ssl
ubuntu-12.04 apache::default apache::ssl
centos-6.4 apache::default apache::ssl
.kitchen.yml
---
driver:
name: vagrant
provisioner:
name: chef_zero
platforms:
- name: ubuntu-12.04
- name: centos-6.4
- name: ubuntu-14.04
suites:
- name: default
run_list:
- recipe[apache::default]
- name: ssl
run_list:
- recipe[apache::ssl]
default ssl
ubuntu-12.04 apache::default apache::ssl
centos-6.4 apache::default apache::ssl
ubuntu-14.04 apache::default apache::ssl
$
Move to the apache cookbook directory
cd ~/chef-repo/cookbooks/apache
OPEN IN EDITOR:
SAVE FILE!
Update .kitchen.yml
---
driver:
name: docker
provisioner:
name: chef_zero
platforms:
- name: centos-6.5
suites:
- name: default
run_list:
- recipe[apache::default]
attributes:
.kitchen.yml
$
Instance Driver Provisioner Last Action
default-centos-65 Docker ChefZero <Not Created>
List the Test Kitchens
kitchen list
$
-----> Starting Kitchen (v1.2.1)
-----> Creating <default-centos-64>...
Step 0 : FROM centos:centos6
---> 68eb857ffb51
Step 1 : RUN yum clean all
---> Running in cdf3952a3f18
Loaded plugins: fastestmirror
Cleaning repos: base extras libselinux updates
Cleaning up Everything
---> b1cccd25ce55
Removing intermediate container cdf3952a3f18
Step 2 : RUN yum install -y sudo openssh-server openssh-clients which curl
---> Running in 9db69ace459d
Loaded plugins: fastestmirror
Create the kitchen
kitchen create
Kitchen created
$
kitchen@localhost's password:
Login to the kitchen
kitchen login
$
kitchen@localhost's password:
Login to the kitchen
kitchen login
kitchen
$
kitchen@localhost's password:
Last login: Wed Sep 24 04:30:29 2014 from 172.17.42.1
Login to the kitchen
kitchen login
kitchen
Kitchen login
Kitchen login
[chef@ip-172-31-44-173 apache]$ kitchen login
Kitchen login
[chef@ip-172-31-44-173 apache]$ kitchen login
ssh
Kitchen login
[chef@ip-172-31-44-173 apache]$ kitchen login
[kitchen@5379d310dc59 ~]$
ssh
Chef client success status
• Requirements to verify chef-client success:
• A target server running the same OS as production
• A chef-client with access to the cookbook
Lab – Apply our policy
• Problem: We have not applied our policy to the
test environment.
• Success Criteria: The default apache recipe
will be applied in the test environment
$
logout
Connection to localhost closed.
Leave the kitchen
exit
$
Go to the right place
cd ~/chef-repo/cookbooks/apache
$
-----> Starting Kitchen (v1.2.1)
-----> Converging <default-centos-64>...
Preparing files for transfer
Resolving cookbook dependencies with Berkshelf 3.1.5...
Removing non-cookbook files before transfer
-----> Installing Chef Omnibus (true)
downloading https://www.getchef.com/chef/install.sh
to file /tmp/install.sh
trying curl...
Apply our policy
kitchen converge
Kitchen converge
Install Chef
Upload cookbooks
Apply the run_list
Questions to ask when testing
ü  Did chef-client complete successfully?
• Did the recipe put the node in the desired state?
• Are the resources properly defined?
• Does the code following our style guide?
Verifying node state
Serverspec
Chef Testing
ü  Did chef-client complete successfully?
• Did the recipe put the node in the desired state?
• Are the resources properly defined?
• Does the code following our style guide?
$
kitchen@localhost's password:
Manually inspect the test node
kitchen login
$
kitchen@localhost's password:
Manually inspect the test node
kitchen login
kitchen
$
kitchen@localhost's password:
Last login: Wed Sep 24 04:30:29 2014 from 172.17.42.1
Manually inspect the test node
kitchen login
kitchen
$
curl: (7) couldn't connect to host
Manually inspect the test node
curl http://localhost
Kitchen login
[chef@ip-172-31-44-173 apache]$ kitchen login
[kitchen@5379d310dc59 ~]$ curl http://localhost
curl: (7) couldn't connect to host
ssh
Lab – Verify node state
• Problem: Manually verifying the state of the test
node is tedious and error-prone.
• Success Criteria: The end state of the node is
automatically tested.
Serverspec
• Write tests to verify your servers
• Not dependent on Chef
• Defines many resource types
• package, service, user, etc.
• Works well with Test Kitchen
• http://serverspec.org/
$
logout
Connection to localhost closed.
Leave the Kitchen
exit
$
Move to the proper directory
cd ~/chef-repo/cookbooks/apache
OPEN IN EDITOR:
SAVE FILE!
Write a Serverspec test
require 'serverspec'
set :backend, :exec
describe 'apache' do
end
test/integration/default/serverspec/default_spec.rb
Generic Expectation Form
describe "<subject>" do
it "<description>" do
expect(thing).to eq result
end
end
OPEN IN EDITOR:
SAVE FILE!
Awesome Expectations
require 'serverspec'
set :backend, :exec
describe "apache::default" do
it "is awesome" do
expect(true).to eq true
end
end
test/integration/default/serverspec/default_spec.rb
$
-----> Running serverspec test suite
/opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -I/tmp/
busser/gems/gems/rspec-support-3.1.2/lib:/tmp/busser/gems/gems/rspec-
core-3.1.7/lib /opt/chef/embedded/bin/rspec --pattern /tmp/busser/suites/
serverspec/**/*_spec.rb --color --format documentation --default-path /
tmp/busser/suites/serverspec
apache::default
is awesome
Finished in 0.02823 seconds (files took 0.99875 seconds to load)
1 example, 0 failures
Finished verifying <default-centos-64> (0m5.03s).
Run the serverspec test
kitchen verify
How would you test our criteria?
• We want a custom home page available on the
web.
What is success?
• Package is installed?
• Page is displayed?
• What else?
OPEN IN EDITOR:
SAVE FILE!
Verify package is installed
require 'serverspec'
set :backend, :exec
describe "apache" do
it "is awesome" do
expect(true).to eq true
end
it "is installed" do
expect(package("httpd")).to be_installed
end
end
test/integration/default/serverspec/default_spec.rb
$
apache
is awesome
is installed (FAILED - 1)
Failures:
1) apache is installed
Failure/Error: expect(package("httpd")).to
be_installed
expected Package "httpd" to be installed
/bin/sh -c rpm -q httpd
package httpd is not installed
Exercise the test
kitchen verify
Test is failing, make it pass
• Test-driven development involves
• Write a test to verify something is working
• Watch the test fail
• Write just enough code to make the test pass
• Repeat
OPEN IN EDITOR:
SAVE FILE!
Update our cookbook
package "httpd"
~/chef-reop/cookbooks/apache/recipes/default.rb
$
-----> Converging <default-centos-64>...
Preparing files for transfer
Resolving cookbook dependencies with Berkshelf 3.1.5...
Removing non-cookbook files before transfer
Transfering files to <default-centos-64>
[2014-11-10T09:20:26+00:00] INFO: Starting chef-zero on host localhost, port 8889
with repository at repository at /tmp/kitchen
One version per cookbook
[2014-11-10T09:20:26+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 11.16.4
[2014-11-10T09:20:27+00:00] INFO: *** Chef 11.16.4 ***
[2014-11-10T09:20:27+00:00] INFO: Chef-client pid: 571
...
Converge the node again
kitchen converge
$
apache
is awesome
is installed
Finished in 0.48165 seconds (files took 1.05
seconds to load)
2 examples, 0 failures
Finished verifying <default-centos-64>
(0m5.64s).
-----> Kitchen is finished. (0m11.84s)
Exercise the test
kitchen verify
What else will you test?
• Is the service running?
• Is the port accessible?
• Is the expected content being served?
• Make sure everything works from a fresh
kitchen, too!
Time to hack!
https://www.flickr.com/photos/peterpearson/424047087
OPEN IN EDITOR:
SAVE FILE!
Extend the Serverspec test
describe 'apache' do
it "is installed" do
expect(package 'httpd').to be_installed
end
it "is running" do
expect(service 'httpd').to be_running
end
it "is listening on port 80" do
expect(port 80).to be_listening
end
it "displays a custom home page" do
expect(command("curl localhost").stdout).to match /hello/
end
end
test/integration/default/serverspec/default_spec.rb
$
apache
is installed
is running
is listening on port 80
displays a custom home page
Finished in 0.3968 seconds
4 examples, 0 failures
Finished verifying <default-centos-64> (0m4.25s).
Verify the kitchen
kitchen verify
Kitchen Workflow
• kitchen create
• kitchen converge
• kitchen verify
• kitchen destroy
• All at once with kitchen test
Chef Testing
ü  Did chef-client complete successfully?
ü  Did the recipe put the node in the desired
state?
• Are the resources properly defined?
• Does the code following our style guide?
Now for our new mandate
• Update the tests
• Watch them fail
• Update the policy
• See tests pass
• Roll-out changes to production
Even Faster Feedback
ChefSpec
Chef Testing
ü  Did chef-client complete successfully?
ü  Did the recipe put the node in the desired
state?
• Are the resources properly defined?
• Does the code following our style guide?
This is too slow!
• To test our code, we need to spin up a test
kitchen, converge a node, execute some tests.
• Our simple test case takes about 2 minutes to
fully execute.
Properly configured resources
• We need a way to verify that the resources in
our recipes are properly configured
• We want to get faster feedback
Lab – Verify the resources
• Problem: We should be able to catch errors
before we need to converge a node
• Success Criteria: Catch a typo prior to
converge
ChefSpec
• Test before you
converge
• Get feedback on
cookbook changes
without the need for
target servers
http://sethvargo.github.io/chefspec/
$
Make a directory for our ChefSpec tests
cd ~/chef-repo/cookbooks/apache
OPEN IN EDITOR:
SAVE FILE!
Write a ChefSpec test
require 'spec_helper'
describe 'apache::default' do
context 'When all attributes are default, on an unspecified platform' do
let(:chef_run) do
runner = ChefSpec::ServerRunner.new
runner.converge(described_recipe)
end
it 'converges successfully' do
chef_run # This should not raise an error
end
end
end
spec/unit/recipes/default_spec.rb
$
.
Finished in 0.00865 seconds (files took 5.5 seconds to load)
1 example, 0 failures
Run the ChefSpec tests
rspec spec/unit/*.rb
OPEN IN EDITOR:
SAVE FILE!
Break the cookbook
package "http"
service "httpd" do
action :start
end
template "/var/www/html/index.html" do
source "index.html.erb"
end
recipes/default.rb
$
F
Failures:
1) apache::default installs apache
Failure/Error: expect(chef_run).to install_package('httpd')
expected "package[httpd]" with action :install to be in Chef run. Other package resources:
package[http]
# ./spec/unit/default_spec.rb:9:in `block (2 levels) in <top (required)>'
Finished in 0.00847 seconds (files took 4.85 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/unit/default_spec.rb:8 # apache::default installs apache
Run the ChefSpec tests
rspec spec/unit/*.rb
OPEN IN EDITOR:
SAVE FILE!
Fix the cookbook
package "httpd"
service "httpd" do
action :start
end
template "/var/www/html/index.html" do
source "index.html.erb"
end
recipes/default.rb
Time to hack!
https://www.flickr.com/photos/peterpearson/424047087
Chef Testing
ü  Did chef-client complete successfully?
ü  Did the recipe put the node in the desired
state?
ü  Are the resources properly defined?
• Does the code following our style guide?
Clean code
Follow best practices, avoid mistakes
Foodcritic
• Check cookbooks for
common problems
• Style, correctness,
deprecations, etc.
• Included with ChefDK
http://www.foodcritic.io/
OPEN IN EDITOR:
SAVE FILE!
Change our recipe
package_name = "httpd"
package "#{package_name}"
service "httpd" do
action :start
end
template "/var/www/html/index.html" do
source "index.html.erb"
end
recipes/default.rb
$
FC002: Avoid string interpolation
where not required: ./recipes/
default.rb:7
Run Foodcritic
foodcritic .
Chef Testing
ü  Did chef-client complete successfully?
ü  Did the recipe put the node in the desired
state?
ü  Are the resources properly defined?
ü  Does the code following our style guide?
Wrapping Up
We’ve only scratched the surface
https://www.chef.io/chef/
Build Anything
• Simple internal applications
• Complex external
applications
• Workstations
• Hadoop clusters
• IaaS infrastructure
• PaaS infrastructure
• SaaS applications
• Storage systems
• You name it
http://www.flickr.com/photos/hyku/245010680/
And Manage it Simply
• Automatically
reconfigure everything
• Linux, Windows, Unixes,
BSDs
• Load balancers
• Metrics collection
systems
• Monitoring systems
• Cloud migrations
become trivial
http://www.flickr.com/photos/helico/404640681/
What questions do you have?
• Ask me anything!
• @nathenharvey
• nharvey@chef.io
• Thank you!

Mais conteúdo relacionado

Mais procurados

Chef vs Puppet vs Ansible vs Saltstack | Configuration Management Tools | Dev...
Chef vs Puppet vs Ansible vs Saltstack | Configuration Management Tools | Dev...Chef vs Puppet vs Ansible vs Saltstack | Configuration Management Tools | Dev...
Chef vs Puppet vs Ansible vs Saltstack | Configuration Management Tools | Dev...
Simplilearn
 

Mais procurados (20)

Automated Infrastructure Testing
Automated Infrastructure TestingAutomated Infrastructure Testing
Automated Infrastructure Testing
 
Journée DevOps : De l'intégration continue au déploiement continu avec Jenkins
Journée DevOps : De l'intégration continue au déploiement continu avec JenkinsJournée DevOps : De l'intégration continue au déploiement continu avec Jenkins
Journée DevOps : De l'intégration continue au déploiement continu avec Jenkins
 
OSDC2014: Testing Server Infrastructure with #serverspec
OSDC2014: Testing Server Infrastructure with #serverspecOSDC2014: Testing Server Infrastructure with #serverspec
OSDC2014: Testing Server Infrastructure with #serverspec
 
Chef vs Puppet vs Ansible vs SaltStack | Configuration Management Tools Compa...
Chef vs Puppet vs Ansible vs SaltStack | Configuration Management Tools Compa...Chef vs Puppet vs Ansible vs SaltStack | Configuration Management Tools Compa...
Chef vs Puppet vs Ansible vs SaltStack | Configuration Management Tools Compa...
 
Automated Deployments with Ansible
Automated Deployments with AnsibleAutomated Deployments with Ansible
Automated Deployments with Ansible
 
Operating Docker
Operating DockerOperating Docker
Operating Docker
 
Introduction to Chef: Automate Your Infrastructure by Modeling It In Code
Introduction to Chef: Automate Your Infrastructure by Modeling It In CodeIntroduction to Chef: Automate Your Infrastructure by Modeling It In Code
Introduction to Chef: Automate Your Infrastructure by Modeling It In Code
 
Nike pop up habitat
Nike pop up   habitatNike pop up   habitat
Nike pop up habitat
 
Using Chef InSpec for Infrastructure Security
Using Chef InSpec for Infrastructure SecurityUsing Chef InSpec for Infrastructure Security
Using Chef InSpec for Infrastructure Security
 
Chef Workflow Demo
Chef Workflow DemoChef Workflow Demo
Chef Workflow Demo
 
Chef vs Puppet vs Ansible vs Saltstack | Configuration Management Tools | Dev...
Chef vs Puppet vs Ansible vs Saltstack | Configuration Management Tools | Dev...Chef vs Puppet vs Ansible vs Saltstack | Configuration Management Tools | Dev...
Chef vs Puppet vs Ansible vs Saltstack | Configuration Management Tools | Dev...
 
London Community Summit 2016 - Fresh New Chef Stuff
London Community Summit 2016 - Fresh New Chef StuffLondon Community Summit 2016 - Fresh New Chef Stuff
London Community Summit 2016 - Fresh New Chef Stuff
 
Prescriptive Security with InSpec - All Things Open 2019
Prescriptive Security with InSpec - All Things Open 2019Prescriptive Security with InSpec - All Things Open 2019
Prescriptive Security with InSpec - All Things Open 2019
 
Compliance Automation with Inspec Part 2
Compliance Automation with Inspec Part 2Compliance Automation with Inspec Part 2
Compliance Automation with Inspec Part 2
 
DevSecCon London 2017: Permitting agility whilst enforcing security by Alina ...
DevSecCon London 2017: Permitting agility whilst enforcing security by Alina ...DevSecCon London 2017: Permitting agility whilst enforcing security by Alina ...
DevSecCon London 2017: Permitting agility whilst enforcing security by Alina ...
 
Compliance Automation Workshop
Compliance Automation WorkshopCompliance Automation Workshop
Compliance Automation Workshop
 
Adding Security to Your Workflow With InSpec - SCaLE17x
Adding Security to Your Workflow With InSpec - SCaLE17xAdding Security to Your Workflow With InSpec - SCaLE17x
Adding Security to Your Workflow With InSpec - SCaLE17x
 
Testing servers like software
Testing servers like softwareTesting servers like software
Testing servers like software
 
Rundeck + Nexus (from Nexus Live on June 5, 2014)
Rundeck + Nexus (from Nexus Live on June 5, 2014)Rundeck + Nexus (from Nexus Live on June 5, 2014)
Rundeck + Nexus (from Nexus Live on June 5, 2014)
 
Splunk: Forward me the REST of those shells
Splunk: Forward me the REST of those shellsSplunk: Forward me the REST of those shells
Splunk: Forward me the REST of those shells
 

Destaque

Destaque (20)

ContainerDays NYC 2016: "The Secure Introduction Problem: Getting Secrets Int...
ContainerDays NYC 2016: "The Secure Introduction Problem: Getting Secrets Int...ContainerDays NYC 2016: "The Secure Introduction Problem: Getting Secrets Int...
ContainerDays NYC 2016: "The Secure Introduction Problem: Getting Secrets Int...
 
Docker Security: Are Your Containers Tightly Secured to the Ship?
Docker Security: Are Your Containers Tightly Secured to the Ship?Docker Security: Are Your Containers Tightly Secured to the Ship?
Docker Security: Are Your Containers Tightly Secured to the Ship?
 
Docker Security workshop slides
Docker Security workshop slidesDocker Security workshop slides
Docker Security workshop slides
 
How GitLab and HackerOne help organizations innovate faster without compromis...
How GitLab and HackerOne help organizations innovate faster without compromis...How GitLab and HackerOne help organizations innovate faster without compromis...
How GitLab and HackerOne help organizations innovate faster without compromis...
 
Atomic CLI scan
Atomic CLI scanAtomic CLI scan
Atomic CLI scan
 
Docker London: Container Security
Docker London: Container SecurityDocker London: Container Security
Docker London: Container Security
 
Practical Approaches to Container Security
Practical Approaches to Container SecurityPractical Approaches to Container Security
Practical Approaches to Container Security
 
Understanding container security
Understanding container securityUnderstanding container security
Understanding container security
 
Open Source Tools for Container Security and Compliance @Docker LA Meetup 2/13
Open Source Tools for Container Security and Compliance @Docker LA Meetup 2/13Open Source Tools for Container Security and Compliance @Docker LA Meetup 2/13
Open Source Tools for Container Security and Compliance @Docker LA Meetup 2/13
 
Monetising Your Skill
Monetising Your SkillMonetising Your Skill
Monetising Your Skill
 
Veer's Container Security
Veer's Container SecurityVeer's Container Security
Veer's Container Security
 
Why You Need to Rethink Container Security
Why You Need to Rethink Container SecurityWhy You Need to Rethink Container Security
Why You Need to Rethink Container Security
 
AWS Security Architecture - Overview
AWS Security Architecture - OverviewAWS Security Architecture - Overview
AWS Security Architecture - Overview
 
AWS Security Best Practices and Design Patterns
AWS Security Best Practices and Design PatternsAWS Security Best Practices and Design Patterns
AWS Security Best Practices and Design Patterns
 
Security best practices for kubernetes deployment
Security best practices for kubernetes deploymentSecurity best practices for kubernetes deployment
Security best practices for kubernetes deployment
 
Monitoring, Logging and Tracing on Kubernetes
Monitoring, Logging and Tracing on KubernetesMonitoring, Logging and Tracing on Kubernetes
Monitoring, Logging and Tracing on Kubernetes
 
London HUG 19/5 - Kubernetes and vault
London HUG 19/5 - Kubernetes and vaultLondon HUG 19/5 - Kubernetes and vault
London HUG 19/5 - Kubernetes and vault
 
Docker Security - Secure Container Deployment on Linux
Docker Security - Secure Container Deployment on LinuxDocker Security - Secure Container Deployment on Linux
Docker Security - Secure Container Deployment on Linux
 
Container Orchestration Wars
Container Orchestration WarsContainer Orchestration Wars
Container Orchestration Wars
 
Docker Security Overview
Docker Security OverviewDocker Security Overview
Docker Security Overview
 

Semelhante a Introduction to Infrastructure as Code & Automation / Introduction to Chef

Stress Free Deployment - Confoo 2011
Stress Free Deployment  - Confoo 2011Stress Free Deployment  - Confoo 2011
Stress Free Deployment - Confoo 2011
Bachkoutou Toutou
 

Semelhante a Introduction to Infrastructure as Code & Automation / Introduction to Chef (20)

Introduction to Infrastructure as Code & Automation / Introduction to Chef
Introduction to Infrastructure as Code & Automation / Introduction to ChefIntroduction to Infrastructure as Code & Automation / Introduction to Chef
Introduction to Infrastructure as Code & Automation / Introduction to Chef
 
Overview of Chef - Fundamentals Webinar Series Part 1
Overview of Chef - Fundamentals Webinar Series Part 1Overview of Chef - Fundamentals Webinar Series Part 1
Overview of Chef - Fundamentals Webinar Series Part 1
 
Introduction to Chef
Introduction to ChefIntroduction to Chef
Introduction to Chef
 
Introduction to Chef - April 22 2015
Introduction to Chef - April 22 2015Introduction to Chef - April 22 2015
Introduction to Chef - April 22 2015
 
Node object and roles - Fundamentals Webinar Series Part 3
Node object and roles - Fundamentals Webinar Series Part 3Node object and roles - Fundamentals Webinar Series Part 3
Node object and roles - Fundamentals Webinar Series Part 3
 
Introduction to Chef
Introduction to ChefIntroduction to Chef
Introduction to Chef
 
Introduction to Chef - Techsuperwomen Summit
Introduction to Chef - Techsuperwomen SummitIntroduction to Chef - Techsuperwomen Summit
Introduction to Chef - Techsuperwomen Summit
 
Chef: Smart infrastructure automation
Chef: Smart infrastructure automationChef: Smart infrastructure automation
Chef: Smart infrastructure automation
 
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...
 
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
Devops continuousintegration and deployment onaws puttingmoneybackintoyourmis...
 
Opscode Webinar: Cooking with Chef on Microsoft Windows
Opscode Webinar: Cooking with Chef on Microsoft WindowsOpscode Webinar: Cooking with Chef on Microsoft Windows
Opscode Webinar: Cooking with Chef on Microsoft Windows
 
Chef Fundamentals Training Series Module 3: Setting up Nodes and Cookbook Aut...
Chef Fundamentals Training Series Module 3: Setting up Nodes and Cookbook Aut...Chef Fundamentals Training Series Module 3: Setting up Nodes and Cookbook Aut...
Chef Fundamentals Training Series Module 3: Setting up Nodes and Cookbook Aut...
 
CouchDB for Web Applications - Erlang Factory London 2009
CouchDB for Web Applications - Erlang Factory London 2009CouchDB for Web Applications - Erlang Factory London 2009
CouchDB for Web Applications - Erlang Factory London 2009
 
Stress Free Deployment - Confoo 2011
Stress Free Deployment  - Confoo 2011Stress Free Deployment  - Confoo 2011
Stress Free Deployment - Confoo 2011
 
.NET Cloud-Native Bootcamp
.NET Cloud-Native Bootcamp.NET Cloud-Native Bootcamp
.NET Cloud-Native Bootcamp
 
Nginx A High Performance Load Balancer, Web Server & Reverse Proxy
Nginx A High Performance Load Balancer, Web Server & Reverse ProxyNginx A High Performance Load Balancer, Web Server & Reverse Proxy
Nginx A High Performance Load Balancer, Web Server & Reverse Proxy
 
Azure handsonlab
Azure handsonlabAzure handsonlab
Azure handsonlab
 
Standardizing and Managing Your Infrastructure - MOSC 2011
Standardizing and Managing Your Infrastructure - MOSC 2011Standardizing and Managing Your Infrastructure - MOSC 2011
Standardizing and Managing Your Infrastructure - MOSC 2011
 
Best Practices for Building WordPress Applications
Best Practices for Building WordPress ApplicationsBest Practices for Building WordPress Applications
Best Practices for Building WordPress Applications
 
Chef for openstack
Chef for openstackChef for openstack
Chef for openstack
 

Mais de Nathen Harvey

Mais de Nathen Harvey (12)

Accelerate Your DevOps Journey
Accelerate Your DevOps JourneyAccelerate Your DevOps Journey
Accelerate Your DevOps Journey
 
Continuous Delivery - GDG Cloud Baltimore
Continuous Delivery - GDG Cloud BaltimoreContinuous Delivery - GDG Cloud Baltimore
Continuous Delivery - GDG Cloud Baltimore
 
Using Error Budgets to Prioritize Work
Using Error Budgets to Prioritize WorkUsing Error Budgets to Prioritize Work
Using Error Budgets to Prioritize Work
 
Introduction to Test Kitchen and InSpec
Introduction to Test Kitchen and InSpecIntroduction to Test Kitchen and InSpec
Introduction to Test Kitchen and InSpec
 
Introduction to Test Kitchen
Introduction to Test KitchenIntroduction to Test Kitchen
Introduction to Test Kitchen
 
Effective Testing with Ansible and InSpec
Effective Testing with Ansible and InSpecEffective Testing with Ansible and InSpec
Effective Testing with Ansible and InSpec
 
Testing Terraform
Testing TerraformTesting Terraform
Testing Terraform
 
DevOps Days India Keynote
DevOps Days India KeynoteDevOps Days India Keynote
DevOps Days India Keynote
 
Compliance Automation with InSpec
Compliance Automation with InSpecCompliance Automation with InSpec
Compliance Automation with InSpec
 
Step AFK: Practical Advice for Career Adavancement
Step AFK: Practical Advice for Career AdavancementStep AFK: Practical Advice for Career Adavancement
Step AFK: Practical Advice for Career Adavancement
 
Walk This Way - An Introduction to DevOps
Walk This Way - An Introduction to DevOpsWalk This Way - An Introduction to DevOps
Walk This Way - An Introduction to DevOps
 
Mongo db at_customink
Mongo db at_custominkMongo db at_customink
Mongo db at_customink
 

Último

Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Último (20)

Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 

Introduction to Infrastructure as Code & Automation / Introduction to Chef

  • 1. Chef Fundamentals by Chef Software, Inc. is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. Intro to Infrastructure as Code Introduction to Chef POSSCON – April 2015 http://bit.ly/posscon2015-chef
  • 2. Nathen Harvey • Community Director at Chef • Co-host of the Food Fight Show • Co-organizer of DevOpsDC meetup • Occasional farmer – http://bit.ly/farmer-nathen • Love Eggs – http://eggs.chef.io • @nathenharvey • nharvey@chef.io
  • 8. Are you experienced? • Experience with Infrastructure as Code or Configuration Management?
  • 9. Are you experienced? • Experience with Infrastructure as Code or Configuration Management? • Experience with Chef?
  • 10. Which version control system do your use?
  • 11. Which version control system do your use? • cp foo foo.bak
  • 12. Which version control system do your use? • cp foo foo.bak • cp foo{,.`date +%Y%m%d%H%M`}
  • 13. Which version control system do your use? • cp foo foo.bak • cp foo{,.`date +%Y%m%d%H%M`} • cp foo{,.`date +%Y%m%d%H%M`-`$USER`}
  • 15. The Sys Admin’s Journey • ssh
  • 16. The Sys Admin’s Journey • ssh • Store notes in ~/server.txt
  • 17. The Sys Admin’s Journey • ssh • Store notes in ~/server.txt • Move notes to the wiki
  • 18. The Sys Admin’s Journey • ssh • Store notes in ~/server.txt • Move notes to the wiki • Write some scripts (setup.sh, fixit.sh, etc.)
  • 19. The Sys Admin’s Journey • ssh • Store notes in ~/server.txt • Move notes to the wiki • Write some scripts (setup.sh, fixit.sh, etc.) • Golden images and snapshots
  • 20. The Sys Admin’s Journey • ssh • Store notes in ~/server.txt • Move notes to the wiki • Write some scripts (setup.sh, fixit.sh, etc.) • Golden images and snapshots • Remote execution via ssh
  • 21. The Sys Admin’s Journey • ssh • Store notes in ~/server.txt • Move notes to the wiki • Write some scripts (setup.sh, fixit.sh, etc.) • Golden images and snapshots • Remote execution via ssh • Policy-driven configuration management
  • 24. Automation Platform • Creates a dependable view of your entire network’s state. • Can handle complex dependencies among the nodes of your network. • Is fault tolerant. • Is secure. • Can handle multiple platforms • Can manage cloud resources • Provides a foundation for innovation
  • 26. Infrastructure as Code • Treat like any other code base
  • 27. Infrastructure as Code • Reconstruct business from code repository, data backup, and compute resources
  • 28. Infrastructure as Code • Programmatically provision and configure components • Treat like any other code base • Reconstruct business from code repository, data backup, and compute resources
  • 29. Policy-based • You capture the policy for your infrastructure in code • A program ensures each node in your infrastructure complies with the policy • A control loop keeps the system stable and allows for change when policy is updated
  • 31. New Compliance Mandate! Graphite Nagios Rails Memcache Postgres Slaves Postgres Master
  • 32. New Compliance Mandate! Graphite Nagios Rails Memcache Postgres Slaves Postgres Master •  Move SSH off of port 22 •  Let’s put it on 2022
  • 33. 6 Golden Images to Update Graphite Nagios Rails Memcache Postgres Slaves Postgres Master 1 3 4 5 6 2 /etc/ssh/sshd_config --- a/sshd_config +++ b/sshd_config -Port 22 +Port 2202
  • 34. 12 Instances to replace Graphite Nagios Rails Memcache Postgres Slaves Postgres Master 1 3 2 5 6 74 8 10 9 11 12 •  Launch •  Delete •  Repeat •  Typically manually
  • 35. Done in maintenance window Graphite Nagios Rails Memcache Postgres Slaves Postgres Master 1 3 2 5 6 74 8 10 9 11 12 •  High stakes •  Late hours •  Risky change
  • 36. New configurations required? Graphite Nagios Rails Memcache Postgres Slaves Postgres Master Do the new instances have new IP Addresses? * Not all connections shown
  • 38. What is Chef • Open source framework for managing complexity in your infrastructure through policy- driven automation code • A community of professionals • A company
  • 40. Chef Server – Policy & State
  • 42. Desired Configuration Node Chef Server chef-client What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]"
  • 43. Desired Configuration Chef Server chef-client What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]"
  • 44. Chef Server – Policy & State
  • 47. HA Proxy Configuration Webservers HA ProxyChef Server Webservers?
  • 48. HA Proxy Configuration Webservers HA ProxyChef Server Webservers?
  • 49. HA Proxy Configuration Webservers HA ProxyChef Server Webservers? { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }
  • 50. HA Proxy Configuration Webservers HA ProxyChef Server Webservers? { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }
  • 51. HA Proxy Configuration Webservers HA ProxyChef Server Webservers? { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" } pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }
  • 52. HA Proxy Configuration Webservers HA Proxy haproxy.cfg server web01 10.1.1.1 weight 1 maxconn 1 check server web02 10.1.1.2 weight 1 maxconn 1 check server web03 10.1.1.3 weight 1 maxconn 1 check server web04 10.1.1.4 weight 1 maxconn 1 check pool_members { "web01" : { "hostname" : "web01", "ipaddress" : "10.1.1.1" }, "web02" : { "hostname" : "web02", "ipaddress" : "10.1.1.2" }, "web03" : { "hostname" : "web03", "ipaddress" : "10.1.1.3" }, "web04" : { "hostname" : "web04", "ipaddress" : "10.1.1.4" }
  • 55. Resources • Piece of the system and its desired state
  • 56. Resources - Package Package that should be installed package "mysql-server" do action :install end
  • 57. Resources - Service Service that should be running and restarted on reboot service "iptables" do action [ :start, :enable ] end
  • 58. Resources - Service File that should be generated file "/etc/motd" do content "Property of Chef Software" end
  • 59. Resources - Cron Cron job that should be configured cron "restart webserver" do hour '2' minute '0' command 'service httpd restart' end
  • 60. Resources - User User that should be managed user "nginx" do comment "Nginx user <nginx@example.com>" uid 500 gid 500 supports :manage_home => true end
  • 61. Resources - DSC DSC resource that should be run dsc_script 'emacs' do code <<-EOH Environment 'texteditor' { Name = 'EDITOR' Value = 'c:emacsbinemacs.exe' } EOH end
  • 62. Resources – Registry Key Registry key that should be created registry_key "HKEY_LOCAL_MACHINE SOFTWAREMicrosoftWindows CurrentVersionPoliciesSystem" do values [{ :name => "EnableLUA", :type => :dword, :data => 0 }] action :create end
  • 63. Resources • Piece of the system and its desired state • http://docs.chef.io/chef/resources.html
  • 64. Lab – Install a text editor • Problem: Our workstation does not have $EDITOR installed • Success Criteria: You can edit files with $EDITOR • $EDITOR is your favorite command line text editor: vim, emacs, or nano
  • 65. What’s up with the card? • http://bit.ly/posscon-workstations • Login: chef • Password: [REDACTED]
  • 66. $ The authenticity of host '54.165.227.226 (54.165.227.226)' can't be established. RSA key fingerprint is c1:ec:ab:66:fb:22:4a: 8f:c2:c5:9b:26:77:f3:dd:b3. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '54.165.227.226' (RSA) to the list of known hosts. chef@54.165.227.226's password: Login to your lab machine ssh chef@54.164.75.30
  • 67. Welcome to your workstation • ChefDK version 0.4.0 is installed • chef --version • Chef user has passwordless sudo access • sudo cat /etc/shadow
  • 68. $ /usr/bin/which: no vim in (/opt/ chefdk/bin:/home/chef/.chefdk/gem/ ruby/2.1.0/bin:/opt/chefdk/embedded/ bin:/usr/local/bin:/bin:/usr/bin:/ usr/local/sbin:/usr/sbin:/sbin:/ home/chef/bin) Is $EDITOR installed? which vim
  • 69. chef-apply • chef-apply is an executable program that allows you to work with resources • Is included as part of the ChefDK • A great way to explore resources • NOT how you’ll eventually use Chef in production
  • 70. $ Recipe: (chef-apply cookbook)::(chef-apply recipe) * package[vim] action install - install version 7.2.411-1.8.el6 of package vim-enhanced Install vim sudo chef-apply -e "package 'vim'"
  • 71. $ Recipe: (chef-apply cookbook)::(chef-apply recipe) * package[emacs] action install - install version 23.1-25.el6 of package emacs Install emacs sudo chef-apply -e "package 'emacs'"
  • 72. $ Recipe: (chef-apply cookbook)::(chef-apply recipe) * package[nano] action install - install version 2.0.9-7.el6 of package nano Install nano sudo chef-apply -e "package 'nano'"
  • 73. Resources • Describe the desired state • Do not need to tell Chef how to get there • What happens if you re-run the chef-apply command?
  • 74. $ Recipe: (chef-apply cookbook)::(chef-apply recipe) * package[vim] action install (up to date) Install $EDITOR again with chef-apply sudo chef-apply -e "package 'vim'"
  • 75. Test and Repair Resources follow a test and repair model package "vim"
  • 76. Test and Repair Resources follow a test and repair model package "vim" Is vim installed?
  • 77. Test and Repair Resources follow a test and repair model package "vim" Is vim installed? Yes
  • 78. Test and Repair Resources follow a test and repair model package "vim" Is vim installed? Done Yes
  • 79. Test and Repair Resources follow a test and repair model package "vim" Is vim installed? Done Yes No
  • 80. Test and Repair Resources follow a test and repair model package "vim" Is vim installed? Done Install it Yes No
  • 81. Test and Repair Resources follow a test and repair model package "vim" Is vim installed? Done Install it Yes No
  • 82. Resources – Test and Repair • Resources follow a test and repair model • Resource currently in the desired state? (test) • Yes – Do nothing • No – Bring the resource into the desired state (repair)
  • 84. OPEN IN EDITOR: SAVE FILE! Hello, Chef! file "/tmp/hello_chef.txt" do content "Hello, Chef" mode "0777" end ~/hello_chef.rb
  • 85. $ Recipe: (chef-apply cookbook)::(chef-apply recipe) * file[/tmp/hello_chef.txt] action create - create new file /tmp/hello_chef.txt - update content in file /tmp/hello_chef.txt from none to 79c290 --- /tmp/hello_chef.txt 2014-10-22 19:59:04.000000000 -0400 +++ /tmp/.hello_chef.txt20141022-23075-19aelx1 2014-10-22 19:59:04.000000000 -0400 @@ -1 +1,2 @@ +Hello, Chef - change mode from '' to '0777' Apply the policy sudo chef-apply hello_chef.rb
  • 86. Resources • Describe the desired state • Do not need to tell Chef how to get there • What happens when you re-apply the policy?
  • 87. $ Recipe: (chef-apply cookbook)::(chef-apply recipe) * file[/tmp/hello_chef.txt] action create (up to date) Apply the policy sudo chef-apply hello_chef.rb
  • 88. Resources • A piece of the system • Its desired state file "/tmp/hello_chef.txt" do content "Hello, Chef" mode "0777" end
  • 89. $ Change the state of the system echo “Hello, #posscon” > /tmp/hello_chef.txt
  • 90. $ Recipe: (chef-apply cookbook)::(chef-apply recipe) * file[/tmp/hello_chef.txt] action create - update content in file /tmp/hello_chef.txt from e453df to 79c290 --- /tmp/hello_chef.txt 2014-10-22 20:00:20.000000000 -0400 +++ /tmp/.hello_chef.txt20141022-23340-17a7m5t 2014-10-22 20:00:50.000000000 -0400 @@ -1,2 +1,2 @@ -“Hello, #posscon” +Hello, Chef Apply the policy sudo chef-apply hello_chef.rb
  • 91. Resources – Test and Repair • Resources use a test and repair model • Resource currently in the desired state? • Yes – Do nothing • No – Bring the resource into the desired state (repair)
  • 93. Recipes • Policy is defined as a collection of resources in recipes. There are lots of abstractions on top of this but resources are the basic building blocks.
  • 94. Install git • The package git should be installed • The file named '/home/chef/.gitconfig' should be created. • It should be owned by the chef user and group. • It should have the content: [user]n name=John Doen email=jdoe@examplen
  • 95. OPEN IN EDITOR: SAVE FILE! Install git package 'git' do action :install end file '/home/chef/.gitconfig' do content "[user]n name=John Doen email=jdoe@examplen" user 'chef' group 'chef' end ~/git.rb
  • 96. $ Recipe: (chef-apply cookbook)::(chef-apply recipe) * package[git] action install - install version 1.7.1-3.el6_4.1 of package git * file[/home/chef/.gitconfig] action create - create new file /home/chef/.gitconfig - update content in file /home/chef/.gitconfig from none to 259950 --- /home/chef/.gitconfig 2014-09-24 00:24:13.558127555 +0000 +++ /tmp/..gitconfig20140924-10180-1ij68vq 2014-09-24 00:24:13.559127555 +0000 @@ -1 +1,4 @@ +[user] + name=John Doe + email=jdoe@example.com - change owner from '' to 'chef' - change group from '' to 'chef' - restore selinux security context Install git sudo chef-apply ~/git.rb
  • 98. Our process • Write policy • Apply policy • Verify policy • Not bad for the simple case, will quickly get untenable
  • 99. Faster Feedback • Speed-up the feedback loops with automated testing. • Have confidence in your changes before you run them in production
  • 100. Chef Testing • Did chef-client complete successfully? • Did the recipe put the node in the desired state? • Are the resources properly defined? • Does the code follow our style guide?
  • 101. Test-driving infrastructure • We are going to use a relatively simple scenario • We are going to explore many facets of testing • We are going to follow a test-first, test-driven model
  • 102. Our Scenario • We want a custom home page available on the web.
  • 103. $ Compiling Cookbooks... Recipe: code_generator::cookbook * directory[/home/chef/chef-repo/cookbooks/apache] action create - create new directory /home/chef/chef-repo/cookbooks/apache * template[/home/chef/chef-repo/cookbooks/apache/metadata.rb] action create_if_missing - create new file /home/chef/chef-repo/cookbooks/apache/metadata.rb - update content in file /home/chef/chef-repo/cookbooks/apache/ metadata.rb from none to 4c0e2d (diff output suppressed by config) * template[/home/chef/chef-repo/cookbooks/apache/README.md] action create_if_missing Create an apache cookbook chef generate cookbook cookbooks/apache
  • 104. Questions to ask when testing • Did chef-client complete successfully? • Did the recipe put the node in the desired state? • Are the resources properly defined? • Does the code following our style guide?
  • 105. Chef client success status • Requirements to verify chef-client success: • A place to store the cookbook artifact
  • 106. Chef client success status • Requirements to verify chef-client success: • A place to store the cookbook artifact • A chef-client with access to the cookbook
  • 107. Chef client success status • Requirements to verify chef-client success: • A place to store the cookbook artifact • A chef-client with access to the cookbook • A target server running the same OS as production
  • 108. Test Kitchen • Test harness to execute code on one or more platforms • Driver plugins to allow your code to run on various cloud and virtualization providers • Includes support for many testing frameworks • Included with ChefDK
  • 109. Test Matrix • Two operating systems ubuntu-12.04 centos-6.4
  • 110. Test Matrix • Two operating systems • One recipe default ubuntu-12.04 apache::default centos-6.4 apache::default
  • 111. Test Matrix • Two operating systems • Two recipes default ssl ubuntu-12.04 apache::default apache::ssl centos-6.4 apache::default apache::ssl
  • 112. Test Matrix • Three operating systems • Two recipes default ssl ubuntu-12.04 apache::default apache::ssl centos-6.4 apache::default apache::ssl ubuntu-14.04 apache::default apache::ssl
  • 113. OPEN IN EDITOR: SAVE FILE! Configuring the Kitchen --- driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] attributes: apache/.kitchen.yml
  • 114. .kitchen.yml • driver - virtualization or cloud provider --- driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] attributes:
  • 115. .kitchen.yml • provisioner - application to configure the node --- driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] attributes:
  • 116. .kitchen.yml • platforms - target operating systems --- driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] attributes:
  • 117. .kitchen.yml • suites - target configurations --- driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] attributes:
  • 118. .kitchen.yml --- driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] default ubuntu-12.04 apache::default centos-6.4 apache::default
  • 119. .kitchen.yml --- driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 suites: - name: default run_list: - recipe[apache::default] - name: ssl run_list: - recipe[apache::ssl] default ssl ubuntu-12.04 apache::default apache::ssl centos-6.4 apache::default apache::ssl
  • 120. .kitchen.yml --- driver: name: vagrant provisioner: name: chef_zero platforms: - name: ubuntu-12.04 - name: centos-6.4 - name: ubuntu-14.04 suites: - name: default run_list: - recipe[apache::default] - name: ssl run_list: - recipe[apache::ssl] default ssl ubuntu-12.04 apache::default apache::ssl centos-6.4 apache::default apache::ssl ubuntu-14.04 apache::default apache::ssl
  • 121. $ Move to the apache cookbook directory cd ~/chef-repo/cookbooks/apache
  • 122. OPEN IN EDITOR: SAVE FILE! Update .kitchen.yml --- driver: name: docker provisioner: name: chef_zero platforms: - name: centos-6.5 suites: - name: default run_list: - recipe[apache::default] attributes: .kitchen.yml
  • 123. $ Instance Driver Provisioner Last Action default-centos-65 Docker ChefZero <Not Created> List the Test Kitchens kitchen list
  • 124. $ -----> Starting Kitchen (v1.2.1) -----> Creating <default-centos-64>... Step 0 : FROM centos:centos6 ---> 68eb857ffb51 Step 1 : RUN yum clean all ---> Running in cdf3952a3f18 Loaded plugins: fastestmirror Cleaning repos: base extras libselinux updates Cleaning up Everything ---> b1cccd25ce55 Removing intermediate container cdf3952a3f18 Step 2 : RUN yum install -y sudo openssh-server openssh-clients which curl ---> Running in 9db69ace459d Loaded plugins: fastestmirror Create the kitchen kitchen create
  • 126. $ kitchen@localhost's password: Login to the kitchen kitchen login
  • 127. $ kitchen@localhost's password: Login to the kitchen kitchen login kitchen
  • 128. $ kitchen@localhost's password: Last login: Wed Sep 24 04:30:29 2014 from 172.17.42.1 Login to the kitchen kitchen login kitchen
  • 132. Kitchen login [chef@ip-172-31-44-173 apache]$ kitchen login [kitchen@5379d310dc59 ~]$ ssh
  • 133. Chef client success status • Requirements to verify chef-client success: • A target server running the same OS as production • A chef-client with access to the cookbook
  • 134. Lab – Apply our policy • Problem: We have not applied our policy to the test environment. • Success Criteria: The default apache recipe will be applied in the test environment
  • 135. $ logout Connection to localhost closed. Leave the kitchen exit
  • 136. $ Go to the right place cd ~/chef-repo/cookbooks/apache
  • 137. $ -----> Starting Kitchen (v1.2.1) -----> Converging <default-centos-64>... Preparing files for transfer Resolving cookbook dependencies with Berkshelf 3.1.5... Removing non-cookbook files before transfer -----> Installing Chef Omnibus (true) downloading https://www.getchef.com/chef/install.sh to file /tmp/install.sh trying curl... Apply our policy kitchen converge
  • 138. Kitchen converge Install Chef Upload cookbooks Apply the run_list
  • 139. Questions to ask when testing ü  Did chef-client complete successfully? • Did the recipe put the node in the desired state? • Are the resources properly defined? • Does the code following our style guide?
  • 141. Chef Testing ü  Did chef-client complete successfully? • Did the recipe put the node in the desired state? • Are the resources properly defined? • Does the code following our style guide?
  • 142. $ kitchen@localhost's password: Manually inspect the test node kitchen login
  • 143. $ kitchen@localhost's password: Manually inspect the test node kitchen login kitchen
  • 144. $ kitchen@localhost's password: Last login: Wed Sep 24 04:30:29 2014 from 172.17.42.1 Manually inspect the test node kitchen login kitchen
  • 145. $ curl: (7) couldn't connect to host Manually inspect the test node curl http://localhost
  • 146. Kitchen login [chef@ip-172-31-44-173 apache]$ kitchen login [kitchen@5379d310dc59 ~]$ curl http://localhost curl: (7) couldn't connect to host ssh
  • 147. Lab – Verify node state • Problem: Manually verifying the state of the test node is tedious and error-prone. • Success Criteria: The end state of the node is automatically tested.
  • 148. Serverspec • Write tests to verify your servers • Not dependent on Chef • Defines many resource types • package, service, user, etc. • Works well with Test Kitchen • http://serverspec.org/
  • 149. $ logout Connection to localhost closed. Leave the Kitchen exit
  • 150. $ Move to the proper directory cd ~/chef-repo/cookbooks/apache
  • 151. OPEN IN EDITOR: SAVE FILE! Write a Serverspec test require 'serverspec' set :backend, :exec describe 'apache' do end test/integration/default/serverspec/default_spec.rb
  • 152. Generic Expectation Form describe "<subject>" do it "<description>" do expect(thing).to eq result end end
  • 153. OPEN IN EDITOR: SAVE FILE! Awesome Expectations require 'serverspec' set :backend, :exec describe "apache::default" do it "is awesome" do expect(true).to eq true end end test/integration/default/serverspec/default_spec.rb
  • 154. $ -----> Running serverspec test suite /opt/chef/embedded/bin/ruby -I/tmp/busser/suites/serverspec -I/tmp/ busser/gems/gems/rspec-support-3.1.2/lib:/tmp/busser/gems/gems/rspec- core-3.1.7/lib /opt/chef/embedded/bin/rspec --pattern /tmp/busser/suites/ serverspec/**/*_spec.rb --color --format documentation --default-path / tmp/busser/suites/serverspec apache::default is awesome Finished in 0.02823 seconds (files took 0.99875 seconds to load) 1 example, 0 failures Finished verifying <default-centos-64> (0m5.03s). Run the serverspec test kitchen verify
  • 155. How would you test our criteria? • We want a custom home page available on the web.
  • 156. What is success? • Package is installed? • Page is displayed? • What else?
  • 157. OPEN IN EDITOR: SAVE FILE! Verify package is installed require 'serverspec' set :backend, :exec describe "apache" do it "is awesome" do expect(true).to eq true end it "is installed" do expect(package("httpd")).to be_installed end end test/integration/default/serverspec/default_spec.rb
  • 158. $ apache is awesome is installed (FAILED - 1) Failures: 1) apache is installed Failure/Error: expect(package("httpd")).to be_installed expected Package "httpd" to be installed /bin/sh -c rpm -q httpd package httpd is not installed Exercise the test kitchen verify
  • 159. Test is failing, make it pass • Test-driven development involves • Write a test to verify something is working • Watch the test fail • Write just enough code to make the test pass • Repeat
  • 160. OPEN IN EDITOR: SAVE FILE! Update our cookbook package "httpd" ~/chef-reop/cookbooks/apache/recipes/default.rb
  • 161. $ -----> Converging <default-centos-64>... Preparing files for transfer Resolving cookbook dependencies with Berkshelf 3.1.5... Removing non-cookbook files before transfer Transfering files to <default-centos-64> [2014-11-10T09:20:26+00:00] INFO: Starting chef-zero on host localhost, port 8889 with repository at repository at /tmp/kitchen One version per cookbook [2014-11-10T09:20:26+00:00] INFO: Forking chef instance to converge... Starting Chef Client, version 11.16.4 [2014-11-10T09:20:27+00:00] INFO: *** Chef 11.16.4 *** [2014-11-10T09:20:27+00:00] INFO: Chef-client pid: 571 ... Converge the node again kitchen converge
  • 162. $ apache is awesome is installed Finished in 0.48165 seconds (files took 1.05 seconds to load) 2 examples, 0 failures Finished verifying <default-centos-64> (0m5.64s). -----> Kitchen is finished. (0m11.84s) Exercise the test kitchen verify
  • 163. What else will you test? • Is the service running? • Is the port accessible? • Is the expected content being served? • Make sure everything works from a fresh kitchen, too!
  • 165. OPEN IN EDITOR: SAVE FILE! Extend the Serverspec test describe 'apache' do it "is installed" do expect(package 'httpd').to be_installed end it "is running" do expect(service 'httpd').to be_running end it "is listening on port 80" do expect(port 80).to be_listening end it "displays a custom home page" do expect(command("curl localhost").stdout).to match /hello/ end end test/integration/default/serverspec/default_spec.rb
  • 166. $ apache is installed is running is listening on port 80 displays a custom home page Finished in 0.3968 seconds 4 examples, 0 failures Finished verifying <default-centos-64> (0m4.25s). Verify the kitchen kitchen verify
  • 167. Kitchen Workflow • kitchen create • kitchen converge • kitchen verify • kitchen destroy • All at once with kitchen test
  • 168. Chef Testing ü  Did chef-client complete successfully? ü  Did the recipe put the node in the desired state? • Are the resources properly defined? • Does the code following our style guide?
  • 169. Now for our new mandate • Update the tests • Watch them fail • Update the policy • See tests pass • Roll-out changes to production
  • 171. Chef Testing ü  Did chef-client complete successfully? ü  Did the recipe put the node in the desired state? • Are the resources properly defined? • Does the code following our style guide?
  • 172. This is too slow! • To test our code, we need to spin up a test kitchen, converge a node, execute some tests. • Our simple test case takes about 2 minutes to fully execute.
  • 173. Properly configured resources • We need a way to verify that the resources in our recipes are properly configured • We want to get faster feedback
  • 174. Lab – Verify the resources • Problem: We should be able to catch errors before we need to converge a node • Success Criteria: Catch a typo prior to converge
  • 175. ChefSpec • Test before you converge • Get feedback on cookbook changes without the need for target servers http://sethvargo.github.io/chefspec/
  • 176. $ Make a directory for our ChefSpec tests cd ~/chef-repo/cookbooks/apache
  • 177. OPEN IN EDITOR: SAVE FILE! Write a ChefSpec test require 'spec_helper' describe 'apache::default' do context 'When all attributes are default, on an unspecified platform' do let(:chef_run) do runner = ChefSpec::ServerRunner.new runner.converge(described_recipe) end it 'converges successfully' do chef_run # This should not raise an error end end end spec/unit/recipes/default_spec.rb
  • 178. $ . Finished in 0.00865 seconds (files took 5.5 seconds to load) 1 example, 0 failures Run the ChefSpec tests rspec spec/unit/*.rb
  • 179. OPEN IN EDITOR: SAVE FILE! Break the cookbook package "http" service "httpd" do action :start end template "/var/www/html/index.html" do source "index.html.erb" end recipes/default.rb
  • 180. $ F Failures: 1) apache::default installs apache Failure/Error: expect(chef_run).to install_package('httpd') expected "package[httpd]" with action :install to be in Chef run. Other package resources: package[http] # ./spec/unit/default_spec.rb:9:in `block (2 levels) in <top (required)>' Finished in 0.00847 seconds (files took 4.85 seconds to load) 1 example, 1 failure Failed examples: rspec ./spec/unit/default_spec.rb:8 # apache::default installs apache Run the ChefSpec tests rspec spec/unit/*.rb
  • 181. OPEN IN EDITOR: SAVE FILE! Fix the cookbook package "httpd" service "httpd" do action :start end template "/var/www/html/index.html" do source "index.html.erb" end recipes/default.rb
  • 183. Chef Testing ü  Did chef-client complete successfully? ü  Did the recipe put the node in the desired state? ü  Are the resources properly defined? • Does the code following our style guide?
  • 184. Clean code Follow best practices, avoid mistakes
  • 185. Foodcritic • Check cookbooks for common problems • Style, correctness, deprecations, etc. • Included with ChefDK http://www.foodcritic.io/
  • 186. OPEN IN EDITOR: SAVE FILE! Change our recipe package_name = "httpd" package "#{package_name}" service "httpd" do action :start end template "/var/www/html/index.html" do source "index.html.erb" end recipes/default.rb
  • 187. $ FC002: Avoid string interpolation where not required: ./recipes/ default.rb:7 Run Foodcritic foodcritic .
  • 188. Chef Testing ü  Did chef-client complete successfully? ü  Did the recipe put the node in the desired state? ü  Are the resources properly defined? ü  Does the code following our style guide?
  • 190. We’ve only scratched the surface https://www.chef.io/chef/
  • 191. Build Anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it http://www.flickr.com/photos/hyku/245010680/
  • 192. And Manage it Simply • Automatically reconfigure everything • Linux, Windows, Unixes, BSDs • Load balancers • Metrics collection systems • Monitoring systems • Cloud migrations become trivial http://www.flickr.com/photos/helico/404640681/
  • 193. What questions do you have? • Ask me anything! • @nathenharvey • nharvey@chef.io • Thank you!