Modern Infrastructure From Scratch - Simon KP, Ecetera

1. Getting it right from the start with Puppet
Modern Infrastructure
From Scratch

2. Senior DevOps Engineer at Ecetera
simon.kp@ecetera.com.au
eskp almost everywhere else
Who am I

3. Overview
● Define, model and provision new infrastructure
● Establish an efficient development workflow
● Get familiar with some of the Puppet concepts

4. Why automate infrastructure?
● Consistently Repeatable and Portable
● Efficient
● Captured Effort
● Executable/Live Self-Documentation

5. Define Modern Infrastructure
● Infrastructure Automation (VirtualBox+Vagrant)
● Version Control (Git+Gitlab)
● Configuration Management (Puppet)
● Test + Deployment Automation (Jenkins+r10k)
● Centralised Log Aggregation (ELK stack)
● Monitoring (Sensu)

6. Infrastructure Model

7. The code
Two repositories:
● Vagrant repo https://github.com/Ecetera/puppet-vagrant
● Control repo https://github.com/Ecetera/puppet-control

8. Puppet Roles & Profiles
Resources > Classes > Modules > Profiles > Role > Node
Allow the business to manage how the infrastructure
looks without defining what it is
Data is abstracted by Hiera

9. Roles definition
# site/role/manifests/init.pp
class role::git {
include profile::jenkins
include profile::rabbit
include profile::mco::client
}
...

10. Profiles definition
# site/profile/manifests/init.pp
class profile::jenkins {
class { '::jenkins': }
jenkins::job { 'puppet-control': }
create_resources(jenkins::credentials, hiera('jenkins_credentials'))
}
...

11. Nodeless Classification
# site.pp
node default {
hiera_include(‘role’)
}

12. Hiera overview
---
# /etc/puppet/hiera.yaml
:yaml:
:datadir: “/etc/puppet/environments/%{::environment}/data”
:hierarchy:
- “%{::hostname}”

13. Hiera overview
---
# data/puppet.yaml
role:
- role::puppetserver
apache::default_vhost: false
...

14. Don’t write modules unless you absolutely have to.
Modules

15. Look for modules
● Puppet Labs approved and supported
● Lots of downloads
● No weird dependencies
● Documentation
● Source is on Github

16. Module Deployment
r10k - manages modules defined in Puppetfile
Deploys Puppet’s dynamic directory environments
based on puppet-control repo’s git branches
Does not resolve modules dependencies

17. Puppetfile
mod ‘puppetlabs/ntp’, ‘3.2.1’
mod ‘rabbitmq’,
:git => https://github.com/puppetlabs/puppetlabs-rabbitmq.git,
:commit => '15e2421048a3495e60482837910441a1248bf6f0'
...

18. Demonstration
http://puppet.eskp.net:5000 Puppetboard view of Puppet
infrastructure
http://control.eskp.net:9090 Jenkins
http://log.eskp.net Kibana dashboard for centralised logs access
http://mon.eskp.net:3000 Uchiwa dashboard overview of Sensu

19. Todo
● Write infrastructure tests
● Address scale
● Provision with puppetlabs-aws module
● Upgrade to Puppet 4

20. Questions?
simon.kp@ecetera.com.au (eskp)
“Modern Infrastructure From Scratch”