Dev to Delivery with Puppet - Sam Bashton, Bashton Ltd.

1. DEV TO DELIVERY WITH
PUPPET
SAM BASHTON, BASHTON LTD

5. HOW DID WE GET HERE?
Previously:
Devs built stuff
Later, Ops came and built production infrastructure
This caused many IT problems
The solution?

6. OPSVELOPMENT

7. DEVOPS

8. WHAT IS DEVOPS REALLY?
Devs doing Ops?
Ops 'coding' infrastructure?
Automating things?
Word that recruiters use without understanding anything
about it?

9. WHAT IS DEVOPS?
BE EXCELLENT TO EACH OTHER

10. WHAT DOES THAT MEAN IN
PRACTICE?

11. WHAT IS OPS?
Working as part of a team to build a reliable environment

12. WHAT IS DEV?
Working as part of a team to build a reliable environment

13. BEING GOOD AT DEV
Follow 'The Twelve Factor App' - http://12factor.net/

14. BEING GOOD AT OPS
Provide consistency across all environments - including
local dev
Provide developers the means to understand what is
happening
Provide as much visibility of everything to everybody

15. PEP20
'Simple is better than complex'
'Complex is better than complicated'
http://legacy.python.org/dev/peps/pep-0020/

16. PROVIDING VISIBILITY
All infrastructure work (Puppet, CloudFormation, etc)
should be checked in to a repository available to the
whole team (Devs + Ops)
Make it easy to see and search logs from all environments
Give as many people as possible access to these logs

17. DEVELOPMENT

18. WHAT AND WHY?
Development environments need to match production as
closely as possible
Builds confidence that something working in dev will work
in production

19. PUPPET EVERYWHERE
Puppet should be used everywhere in the dev and
deployment process
Production
Staging
Integration environments
Test environments
Local dev machines

20. PUPPET CONFIG DOGMA
The same Puppet manifests and modules should be
deployable to all environments without any modification

21. PUPPET CONFIG DOGMA
if statements in manifests are a 'bad smell' and should be
avoided as much as possible

22. PUPPET APPLICATION
CONFIG DOGMA
Separate config files per environment are a 'bad smell' too
Avoid manifests that look like below:
file { '/etc/nginx/nginx.conf':
source => "puppet:///localmodules/data/nginx/${hostname}.conf",
}
Make it easy to 'miss' replicating things between
environments, or make mistakes

23. VAGRANT
Builds virtual machines from Puppet manifests
Makes it easy to spin up short-lived dev instances
Quick to get working
Avoid ops being a blocker for dev

24. VAGRANT + DOCKER
Reduce dev environment spin-up time
Docker makes it easier to create more realistic
environments
Docker images for drop-in use with Vagrant available:
https://github.com/BashtonLtd/docker-vagrant-images

25. BETTER MATCH LIVE
ENVIRONMENTS

26. ONE SET OF MANIFESTS,
MANY ENVIRONMENTS
Different environments need different config
Resource locations
Settings

27. DEALING WITH DIFFERING
ENVIRONMENTS
Hiera
Allows separation of logic from data
Put anything that differers by environment in a separate
file
Combine with custom facts

28. HIERA.YAML
:hierarchy:
- env/%{envname}
- services/%{service}
- common

29. CUSTOM FACTS IN VAGRANT
config.vm.provision :puppet do |puppet|
puppet.manifests_path = "puppet/manifests"
puppet.manifest_file = "site.pp"
puppet.module_path = ["puppet/localmodules","puppet/modules"]
puppet.hiera_config_path = "puppet/hiera.yaml"
puppet.facter = {
"envname" => "vagrant",
"service" => "web",
}
end

30. CUSTOM FACTS ON MACHINES
Drop a file into /etc/facter/facts.d
service=web
envname=stage

31. HIERA IN ACTION
env/vagrant.yaml:
web::hostname: vagrantdev.local
env/stage.yaml:
web::hostname: stage.example.com

32. HIERA IN ACTION
common.yaml:
postfix::server::relayhost: '[mailtrap.io]:2525'
env/live.yaml:
postfix::server::relayhost: 'email-smtp.eu-west-1.amazonaws.com:587'

33. PRODUCTION

34. 'WORKED IN DEV'
Devs and ops need the right data to be able to debug
If only ops have access the the data, how much can devs
really help?

35. PROVIDING VISIBILITY
Metrics and Service Health
Logging
Dashboards

36. METRICS AND SERVICE
HEALTH
Sensu
Health checks
Collection of statistics and export to Graphite

37. CENTRALISED LOGGING
Variety of approaches available
Logstash
Graylog2

38. GRAYLOG2
Simple to get up and running
Puppet module available:
https://forge.puppetlabs.com/graylog2/graylog2

41. VISIBILITY
Dashboards show (near) realtime metrics let everyone see
the current state of the system
Don't just include system data - business metrics add
context