Continuous deployment of puppet modules

CONTINUOUS
DEPLOYMENT OF PUPPET
MODULES
HOW WE DO IT AT MAILCHIMP

EMAIL SERVICE PROVIDER
Deliver ~500 million emails daily
723 million emails delivered on Cyber Monday
of 97Sender Score
http://delivery.mailchimp.com/

Image source: https://blog.engineyard.com/2014/con gure-
before-you-boot

CONFIGURATION MANAGEMENT IS HARD
"With Chef, Puppet, and CFEngine we found a
not-insigni cant learning curve on setting up
the different server daemons and learning the
DSL. This was particularly challenging when
we were con guring unique software not yet
given recipes by the existing community.
Given our cluster sizes, we also didn't really
need any of the advanced features those
systems provided."
- README from internally built tool

MOVE TO COLOCATION
Buy vs. Lease analogy
Grow our Operations team
Needed a tool with dry-run mode

HOW DO WE CATCH THESE MISTAKES AS EARLY
AS POSSIBLE?
AUTONOMATION
"automation with a human touch"
1. Detect the abnormality
2. Stop
3. Fix or correct the immediate condition

HOW DO WE CATCH THESE MISTAKES AS EARLY
AS POSSIBLE?
DSL tools
Editor Support
Source Code Management
Continuous Integration

DSL TOOLS
Puppet
ERB
YAML
Puppet Style Guide

PUPPET
puppet parser validate mymanifest.pp

package { 'openssh-server':
ensure => installed,
}
file { '/etc/ssh/sshd_config':
source => 'puppet:///modules/sshd/sshd_config',
owner => 'root',
group => 'root',
mode => '640',
notify => Service['sshd'] /* sshd will restart whenever you
edit this file. */
require => Package['openssh-server'],
}
service { 'sshd':
ensure => running,
enable => 'true',
hasstatus => 'true',
hasrestart => 'true',
}

$ puppet parser validate validate_1.pp
Error: Could not parse for environment production:
Syntax error at 'require'; expected '}' at validate_1.pp:12

ERB
erb -P -x -T '-' mytemplate.erb | ruby -c

restrict default kod nomodify notrap nopeer<% unless @service %> noquery
restrict 127.0.0.1
restrict -6 ::1
driftfile /var/lib/ntp/drift
<% @serverlist.sort.each do |server| -%>
server <%= server %> iburst maxpoll 6
restrict <%= server %> mask 255.255.255.255 nomodify notrap noquery
<% end -%>

$ erb -P -x -T '-' broken-ntp.conf.erb | ruby -c
-:11: syntax error, unexpected $end, expecting kEND

YAML
npm install -g js-yaml; js-yaml hiera.yaml
ruby -e "require 'yaml'; YAML.load_file('hiera.yaml')"

---
ntp::servers:
- 0.us.pool.ntp.org
- 1.us.pool.ntp.org
- 2.us.pool.ntp.org
- 3.us.pool.ntp.org
hp::ilo::settings:
ssh_status
type: global
value: true
ssh_port
type: global
value: '22'
http_port
type: global
value: '80'
https_port
type: global
value: '443'

$ js-yaml hiera.yaml
JS-YAML: bad indentation of a mapping entry at line 9, column 13:
type: global
^
$ ruby -e "require 'yaml'; YAML.load_file('hiera.yaml')"
yaml.rb:133:in `load':
syntax error on line 9, col 14: ` value: true' (ArgumentError)
from yaml.rb:133:in `load'
from yaml.rb:144:in `load_file'
from yaml.rb:143:in `open'
from yaml.rb:143:in `load_file'
from -e:1

PUPPET STYLE GUIDE
https://docs.puppetlabs.com/guides/style_guide.html
gem install puppet-lint
puppet-lint --fix /my/puppet/code

}
owner => 'root',
group => 'root',
mode => '640',
notify => Service['sshd'], /* sshd will restart whenever you
edit this file. */
}
service { 'sshd':
ensure => running,
enable => 'true',
hasstatus => 'true',
hasrestart => 'true',
}

$ puppet-lint validate_2.pp
WARNING: quoted boolean value found on line 16
WARNING: indentation of => is not properly aligned on line 6
WARNING: mode should be represented as a 4 digit octal value
or symbolic mode on line 9
WARNING: /* */ comment found on line 10

$ puppet-lint --fix validate_2.pp
FIXED: quoted boolean value found on line 16
FIXED: indentation of => is not properly aligned on line 6
FIXED: mode should be represented as a 4 digit octal value
or symbolic mode on line 9
FIXED: /* */ comment found on line 10

}
owner => 'root',
group => 'root',
mode => '0640',
notify => Service['sshd'], # sshd will restart whenever you
# edit this file.
}
service { 'sshd':
ensure => running,
enable => true,
hasstatus => true,
hasrestart => true,
}

--- validate_2.pp 2014-12-08 09:43:38.000000000 -0500
+++ validate_2.pp-fixed 2014-12-08 09:50:51.000000000 -0500
@@ -3,18 +3,18 @@
}
- source => 'puppet:///modules/sshd/sshd_config',
- owner => 'root',
- group => 'root',
- mode => '640',
- notify => Service['sshd'], /* sshd will restart whenever you
- edit this file. */
+ source => 'puppet:///modules/sshd/sshd_config',
+ owner => 'root',
+ group => 'root',
+ mode => '0640',
+ notify => Service['sshd'], # sshd will restart whenever you
+ # edit this file.
}
service { 'sshd':
ensure => running,
- enable => 'true',
- hasstatus => 'true',
- hasrestart => 'true',
+ enable => true,
+ hasstatus => true,
+ hasrestart => true,
}

VIM
PLUGINS FTW
with
Syntastic
vim-puppet
UltiSnips vim-snippets

EMACS
http://www.emacswiki.org/emacs/PuppetProgramming

GEPPETTO
http://puppetlabs.github.io/geppetto/index.html

COMMIT HOOKS
SCRIPT RUNNING THE DSL TOOLS AGAINST NEW FILES

JENKINS
HTTPS://GITHUB.COM/VSTONE/JENKINS-
PUPPET-SCRIPTS

REMEMBER TRUNK BASED
DEPLOYMENT?
# Keep environment up-to-date
vcsrepo { '/etc/puppet/environments/production':
ensure => latest,
provider => hg,
source => 'https://localhost/mercurial/puppet-modules',
}

REVIEW TIME!
Catch mistakes early
Automation with a human touch
Trunk Based Deployments

Slide sources at
THANKS!
BILL O'NEILL
WONEILL@POBOX.COM
@WONEILL
http://github.com/woneill/puppetcamp_atlanta_2014

Continuous deployment of puppet modules

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (11)

Semelhante a Continuous deployment of puppet modules

Semelhante a Continuous deployment of puppet modules (20)

Último

Último (20)

Continuous deployment of puppet modules