Puppet getting started by Dirk Götz

www.netways.de // blog.netways.de // @netways

PUPPET GETTING STARTED
27 NOVEMBER 2013 | PUPPET CAMP

DIRK GÖTZ | NETWAYS GMBH

Make IT do more with less


AGENDA

￭ Brief introduction
￭ Configuration management
￭ Components
￭ Design your environment
￭ Design your workflow
￭ Design your module



BRIEF INTRODUCTION



BRIEF INTRODUCTION TO NETWAYS
• Founded in 1995
• Open source since 1997

• 40 employees

• Specialised in open source systems
management and open source data
center infrastructure



NETWAYS CONFERENCES
Puppet Camp 2014
• 11 April 2014
• 110 attendees (April 2013)
• 170 attendees (November 2014)

Open Source Data Center Conference
• 8 – 10 April 2014
• 120 attendees (2013)
• 2 tracks of presentations & workshops



CONFIGURATION MANAGEMENT



CONFIGURATION MANAGEMENT

Types of „Automation“
Manual configuration

Golden images

Self-made scripts

Software solutions

for i in $(cat host.cfg)
do
ssh user@$i uname -a
done


THE PUPPET WAY
1. Define

2. Simulate
4. Report

3. Enforce
Current State

Desired State



THE PUPPET WAY - DEFINE

￭ Using Puppet Domain Specific Language (DSL)
￭ Describe relationship between resources
￭ Create modular reuseable code



THE PUPPET WAY - SIMULATE

￭ Simulate deployment
￭ Without changes to your system
￭ Test and fix it before enforcing
$ sudo puppet apply ntp.pp --verbose --noop
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Info: Loading facts in /var/lib/puppet/lib/facter/vmware.rb
Info: Loading facts in /var/lib/puppet/lib/facter/last_run.rb
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppi_projects.rb
Info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
Error: Could not find template 'ntp/ntp.conf.erb' at /tmp/ntp.pp:9 on node puppet.localdomain
Error: Could not find template 'ntp/ntp.conf.erb' at /tmp/ntp.pp:9 on node puppet.localdomain



THE PUPPET WAY - ENFORCE

￭ Compares current state and desired state
￭ Changes to desired state

Current State

Desired State

￭ Idempotency
$ sudo puppet agent -t
Info: Retrieving plugin
Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb
Info: Loading facts in /var/lib/puppet/lib/facter/vmware.rb
Info: Loading facts in /var/lib/puppet/lib/facter/last_run.rb
Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/lib/puppet/lib/facter/puppi_projects.rb
Info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb
Info: Caching catalog for puppet.localdomain
Info: Applying configuration version '1384768191‚
Notice: Finished catalog run in 0.54 seconds



THE PUPPET WAY - REPORT

￭ Report changes and metrics
￭ Many different presentation formats



COMPONENTS – BASIC TOOLS



PUPPET WORKFLOW

SSL-Encryption



FACTER

￭ Open source Ruby library by Puppet Labs
￭ Provides information on your host
￭ CLI returns key-value pairs
$ facter
architecture => x86_64
augeasversion => 0.9.0
blockdevice_sr0_model => QEMU DVD-ROM
blockdevice_sr0_size => 1073741312
blockdevice_sr0_vendor => QEMU
blockdevices => sr0,vda
domain => localdomain
facterversion => 1.7.3
filesystems => ext4,iso9660
fqdn => puppet.localdomain
$ facter osfamily
RedHat


PUPPET

￭ Client server based
• REST-API
• X509 certificates

￭ Platform independent
• Puppet Domain Specific Language
• Providers for different platforms



PUPPET DOMAIN SPECIFIC LANGUAGE

￭ Abstracts resources
￭ Resource types
•
•
•
•
•

user, group,
file, package, service,
exec,
…and many more
possible to create your own

￭ Type and title/name pairs must be unique



PUPPET PROVIDERS

￭ Resource Abstraction Layer
• Resource types
• Providers



PUPPET – MORE ABSTRACTION

￭ Class groups resources in one manifest

￭ Syntax constructs possible


PUPPET – SYNTAX CONSTRUCTS

￭ Variables (but are more like constants)
￭ Logical structures
• selectors
• case statements
• if/elseif/else statements

￭ Dependencies
• require/before
• subscribe/notify

￭ Inheritance



PUPPET – MORE ABSTRACTION

￭ Module groups classes and corresponding files
modulename
|-- files – static files
|-- lib – custom facts, functions, resources
|-- manifests – manifests containing classes
|-- spec – tests for rspec-puppet
|-- templates – dynamic files
|-- tests – examples declaring classes



PUPPET FILESERVER

￭ File transfer from master
• puppet:///modules/modulename/filename

￭ “Magic“ mount point for modules
￭ Additional mount points possible
￭ Recursion for directories possible



PUPPET PLUGIN SYNC

￭ Option on the agent
￭ Syncs from all modules
• custom facts
• custom resources (types and providers)

￭ Before the actual Puppet run occurs



PUPPET TEMPLATES

￭ Function executed on server
￭ ERB (Plain text with embedded Ruby)
•
•
•
•
•

variables (including facts)
conditionals
iterations
access to tags and classes
usage of Puppet functions

￭ Combination of multiple templates possible



PUPPET NODE DECLARATION

￭ Connects system (node) with functionality (classes)
• Single hosts

• Mulitiple hosts (by regular expression)

• Default

• Inheritance possible


COMPONENTS – BASIC GUI



BASIC GUI – ADDED FUNCTIONALITY

￭ Reporting target
• Collects reports
• Graphical presentation

￭ External Node Classifier (ENC)
• Graphical node declaration
• Adds groups



PUPPET - REPORTING

￭ Enable on agent
￭ Choose target on server
•
•
•
•
•
•

http/https
log
tagmail
store
rrdgraph
puppetdb

￭ Multiple targets possible
￭ Create your own


PUPPET - ENC

￭ External source for node declaration
• Script returning yaml
• LDAP

￭ Merged with internal node declaration
$/etc/puppet/node.rb puppet.localdomain
--environment: production
parameters:
foreman_env: production
owner_name: Admin User
domainname: ""
owner_email: root@localdomain
root_pw: $1$default$hCkak1kaJPQILNmYbUXhD0
puppetmaster: ""
classes:
profiles::default:

dn: cn=testserver,ou=Hosts,dc=madstop,dc=com
objectClass: device
objectClass: ipHost
objectClass: puppetClient
objectClass: top
cn: testserver
environment: testing
ipHostNumber: 192.168.0.50
description: My test server
l: dc1
puppetClass: testing
puppetVar: owner_name=„Admin user“



PUPPET DASHBOARD

￭ Open source ruby web interface
• Version <= 1.2 by Puppet Labs
• Version >= 2 by community

http://puppetlabs.com/presentations/story-dashboard-20


COMPONENTS – ORCHESTRATION



ORCHESTRATION

￭ Centralized execution on multiple systems
￭ Different solutions
•
•
•
•
•
•

ssh loops
func
fabric
capistrano
mcollective
… and many more



MCOLLECTIVE

￭ Open source Ruby framework by Puppet Labs
￭ Facter for grouping
￭ Middleware for communication
$ mco package status httpd -F kernel=Linux
* [ ============================================================> ] 3 / 3
centos63a
ubuntu1204a
master

version = httpd-2.2.15-15.el6.centos.1
version = httpd-purged
version = httpd-purged

---- package agent summary ---Nodes: 3 / 3
Versions: 1 * 2.2.15-15.el6.centos.1, 2 * purged
Elapsed Time: 0.12 s



PUPPET ENTERPRISE CONSOLE

￭ Ruby web interface based on dashboard
• Auditing
• Live management



COMPONENTS – DEPLOYMENT



DEPLOYMENT

￭ Config management needs running agent
￭ (Agent needs running operating system)
￭ Different solutions:
•
•
•
•
•
•

Kickstart / Autoyast / Preseed / Jumpstart
Cobbler / kickstand
Razor
Puppet Cloud Provisioner
Foreman
… and many more



RAZOR

￭ Open source Ruby library by Puppet Labs/EMC
￭ Client
• micro-kernel for PXE boot

￭ Server
• REST-API
• CLI

￭ Policy connects
• host profiles
• operating system
• config management

{
"name": "centos-for-small",
"repo": { "name": "centos-6.4" },
"installer": { "name": "centos" },
"broker": { "name": "noop" },
"enabled": true,
"hostname": "host${id}.example.com",
"root_password": "secret",
"max_count": "20",
"rule_number": "100",
"tags": [{ "name": "small", "rule": ["<=", ["num", ["fact",
"processorcount"]], 2]}]
}


CLOUD PROVISIONER

￭ Instantiates cloud instances
•
•
•
•

EC2
VMware
OpenStack
Google Compute Engine

￭ Installs Puppet using SSH



FOREMAN

￭ Open source Ruby web interface by Ohad Levy
(Red Hat)
• Reporting target
• ENC
• Provisioning



FOREMAN – SMART PROXIES

￭ Connects GUI and Backends
•
•
•
•
•

puppet
puppetca
tftp
dhcp
dns



FOREMAN – COMPUTE RESOURCES

￭ Integrates virtualization and cloud platforms
•
•
•
•
•
•
•

Libvirt
oVirt / RHEV
VMware
EC2
Google Compute Engine
OpenStack
Rackspace



COMPONENTS – MORE FEATURES



EXPORTED RESOURCES

￭ One node creates resource
￭ Another node realizes that resource
￭ Resource needs to be stored:
• Stored configs (deprecated)
• PuppetDB

￭ Use cases:
•
•
•
•

host entries
sshkey management
monitoring / backup
other centralized services


PUPPETDB

￭ PostgreSQL and Java based data warehouse solution
by Puppet Labs
￭ Collects
• Facts
• Catalogs
• Reports (optional)

￭ Used for
• Inventory service
• Exported resources



HIERA

￭ Hierarchical data look-up separates code and data
￭ Integrated in Puppet >3 / addon for Puppet <3
$cat hiera.yaml
--:backends:
- yaml
:yaml:
:datadir: /etc/puppet/hieradata
:hierarchy:
- `hosts/${::fqdn}´
- `location/${::location}´
- common

$cat hosts/specialhost.localdomain
ntp::server = 192.168.23.23
$ cat location/rz2
ntp::server = 192.168.2.23
yum:mirror = 192.168.2.42
$ cat common
ntp::server = 192.168.0.23
yum::mirror = 192.168.0.42
proxy::server = [`192.168.0.237´,`192.168.0.238´]



COMPONENTS – SOFTWARE MANAGEMENT



SOFTWARE MANAGEMENT

￭ Provider needs defined sources for software
￭ Local mirror reduces traffic
￭ Many different tools
•
•
•
•
•

rsync / createrepo
updian
Spacewalk / Red Hat Satellite / Suse Manager
pulp / katello
… and many more



PULP

￭ Open source Python repository management
by Red Hat
￭ Server:
• Import and upload of content
• rpm and puppet modules
• Publish web-based or ISO images

￭ Client:
• Server-side management and reporting



KATELLO

￭ Java web interface by Red Hat
￭ Combines:
• candlepin (subscription management)
• pulp (software management)
• Foreman + Puppet (configuration management)



DESIGN YOUR ENVIRONMENT



CHOICES

￭ Support / packages needed?
￭ Best method to report?
￭ Resources to export?
￭ Deployment needed?
￭ Software management needed?
￭ Node declaration or ENC?



SCALE UP

￭ Puppet easily scales up
￭ Start simple, grow with your environment
￭ Rule of thumb:
number of nodes * catalog compile time (in seconds)
number of masters = -----------------------------------------------------------------------------cores per master * run interval (in seconds)



SCALE UP

￭ Only one certficate authority
• Only needed for registration

￭ Load balance Puppet traffic
￭ GUI only required for users
• But ENC / Reporting always required

￭ Orchestration



SCALE UP – EXAMPLE: PUPPET ENTERPRISE



DESIGN YOUR WORKFLOW



EDITOR

￭ vim – text editor
•
•
•
•

vim-puppet – synthax highlight
tabular – style guide conformity
puppet-lint – style guide conformity
syntastic – validation

￭ Geppetto – eclipse based IDE
synthax hightlight, style guide conformity, module creation



STYLEGUIDE

￭ Official guide
http://docs.puppetlabs.com/guides/style_guide.html

•
•
•
•

readability
work with and without features
simple and robust
shareable / useable by others

￭ Create your own
• based on official guide
• make your own rules



TESTING

￭ puppet parser validate – syntax
￭ puppet-lint – style guide conformity
￭ puppet apply --noop – simulate
￭ vagrant – simulate
￭ rspec-puppet – expected results



VERSION CONTROL SYSTEM

￭ Pre-Commit / Pre-Receive
• use for validation and review

￭ Post-Commit
• checkout in environment

￭ Versioning for change management
￭ Solutions:
• Version control: svn, git, bazaar
• Validation & Review: gerrit, jenkins/hudson



STAGING

￭ Puppet environments
• Different versions of modules

￭ Use your own versioning
• config_version: script returns version string



DOCUMENTATION

￭ Inline
• RDoc markup
• Console output
• HTML generation

￭ README
• Markdown

￭ Modulefile
• used by Puppet module tool



DESIGN YOUR MODULE



CONSIDERATIONS

￭ Where to start?
￭ Use cases?
•
•
•
•

different platforms
different roles
small adjustments
one-time or multiple objects

￭ ‚Part of it‘ or separate?
• Dependencies

￭ Templates or files?
￭ Write your own?


PUPPET FORGE

￭ Community platform for modules
• Web platform
• Command line tool

￭ Module information:
•
•
•
•
•

Author
Project homepage and issue tracker
Tags
Releases and download count
Test results



PUPPET FORGE – EXAMPLE

example42
￭ 96 modules on forge / more on github
￭ always same layout
￭ supports: Red Hat, CentOS, Fedora, Ubuntu, Debian,
Mint, SLES, OpenSuSE, FreeBSD
￭ adds monitoring, firewall and puppi (own script library)



MODULE DESIGN – MORE ABSTRACTION

￭ Classes abstract resources
￭ Modules abstract classes
￭ Nodes contain the logic

Not maintainable!

node 'basil.puppetlabs.vm' {
class { 'apache’:
version => 'latest’,
}
class { 'motd': }
class { 'ssh’: }
if $::operatingsystem == 'solaris' {
class { 'users’:
default_shell => '/bin/false’,
}
}
else {
class { 'users': }
}
Class['ssh’] -> Class['users’]
}




￭ Profiles contain the logic
• abstract the modules
• separate implementation
from technology

class profiles::application {
include tomcat
include mysql
include componenta
}
class profiles::application::x inherits
profiles::application {
include componentb
componentb::resource { 'name':
ensure => present,
}
}
class profiles::application::y inherits
include componentc
include componentd
}
class profiles::application::z inherits
include componentb
include componentd
include dependency
Class['dependency'] -> Class['componentd']
}



￭ Profiles contain the logic
￭ Roles contain business logic

class role::webapp {
include profiles::base
include profiles::customapp
include profiles::test_tools
}

• no logic, just profiles
• separate business role from implementation

￭ Node has exactly one role
• No Puppet know-how needed
for node declaration

node ‘web1.example.com’ {
include role:webapp
}



QUESTIONS & ANSWERS



QUESTIONS & ANSWERS
NETWAYS GmbH
Deutschherrnstrasse 15-19
90429 Nürmberg

Phone: +49 911 92885-0
Fax: +49 911 92885-77

Email: info@netways.de
Website: www.netways.de
Twitter: twitter.com/netways
Facebook: facebook.com/netways
Blog: blog.netways.de

Puppet getting started by Dirk Götz

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Puppet getting started by Dirk Götz

Semelhante a Puppet getting started by Dirk Götz (20)

Último

Último (20)

Puppet getting started by Dirk Götz