"Oscar: Rapid Iteration with Vagrant and Puppet Enterprise" by Adrien Thebo, Software Engineer, Puppet Labs. Presentation Overview: When trying to debug software problems it's critical to be able to reproduce the original situation, and Puppet Enterprise is no exception to this. The Puppet Labs support team needs a way to rapidly reproduce customer issues across a wide range of operating systems and various versions of Puppet Enterprise. Oscar is a set of Vagrant plugins that handles machine provisioning and configuration to install Puppet Enterprise. It's designed to make building a Puppet Enterprise as simple as running `vagrant up`. While Oscar was originally built for supporting Puppet Enterprise, it provides a general platform for developing and testing against Puppet Enterprise. This talk will go over the history of Oscar, the current state, how it's used, and where to get it. Speaker Bio: Adrien Thebo has been in the Operations/Software development field since 2005, starting at small IT shop in Boise, Idaho. He started at Puppet Labs in 2011 on the Operations team, and used Puppet to run the Puppet Labs infrastructure. In 2013 he transferred to the Community platform team, working with contributors to merge their contributions in Puppet Core. He develops and maintains a number of Puppet modules and tools around Puppet, and when he's not writing code for Puppet then he's probably blogging about it.

1. Oscar
Rapid Iteration with Vagrant
and Puppet Enterprise
Adrien Thebo | Puppet Labs
@nullfinch | Freenode: finch

2. Hi.

3. Introductions n’ stuff
On/Off Operations/Development, ~8 years
Devops before it was cool

4. Introductions n’ stuff
Puppet Labs, 2.5 years
Operations Engineer, 2 years
Community Software Engineer, 6 months

5. Things I do

6. r10k
Better Puppet deployment, powered by robots

7. Puppet modules
puppet-network
puppet-portage
puppet-filemapper

8. So how about that Vagrant?

9. Mandatory audience fun time!
Who has…
heard of Vagrant?
tried Vagrant?
used Vagrant regularly?
developed on Vagrant or plugins?
created Vagrant?

10. A brief introduction to Vagrant

11. Good idea, bad idea
Good idea: hand crafted, one of a kind artisan furniture
Bad idea: hand crafted, one of a kind artisan VMs

12. Before Vagrant
Hand rolled VMs
No automation
Unreproducible
Full of forgotten hacks

13. Vagrant is…
Local VM infrastructure without the pain
Easy to configure
Reproducible
Portable

14. Vagrant Workflow
generate
provision
use/break
discard
It’s your very own cloud!

15. Laying the Groundwork
Operations @ Puppet Labs
… Support @ Puppet Labs?
Infrastructure & client support

16. The problem space

17. Supporting PE 2
Need to reproduce customer problems
Various supported platforms
Various node configurations

18. Supporting PE 2
Speed/reproducibility paramount
Have to meet SLA
Also handle other operations work

19. Support ♥ Vagrant
Receive ticket
Duplicate client environment
Build environment
Reproduce & debug
Help customer^W^Wprofit!!!

20. Limitations
If this presentation was all roses and butterflies
Then it would be very boring.

21. Choose your own adventure
Like any tool, Vagrant makes tradeoffs
Some use cases are easier than others

22. Vagrant excels at…
Static/Stable environments
Configure VM definition
Set up provisioning
git commit -m 'Add Vagrantfile'

23. Vagrant struggles with…
Highly mutable environments
Complex provisioning steps
Configuration reuse
Partial configuration
Distributable configuration

24. The Vagrantfile DSL
Pro - all the power of Ruby
Con - all the power of Ruby

25. A descent into madness
(I’m sorry.)
DONT. DO. THIS.
#vi:setft=ruby:
require'yaml'
begin
#Loadconfig
yaml_config=File.join(File.dirname(__FILE__),"config.yaml")
config=nil
File.open(yaml_config)do|fd|
config=YAML.load(fd.read)
end
nodes =config["nodes"]
profiles =config["profiles"]
nodes.eachdo|node|
#SetdefaultPEconfiguration,andallownodeoverridingofthesevalues
defaults={"pe"=>config['pe']}
node.merge!(defaults)do|key,oldval,newval|
ifoldval.is_a?Hash
newval.mergeoldval
else
warn"Triedtomergehashvalueswith#{key}=>[#{oldval},#{newval}],butwasnotahash.Using#{oldval}."
oldval
end
end
profile =node["profile"]
node.merge!profiles[profile]
end
rescue=>e
puts"Malformedormissingconfig.yaml:#{e}"
putse.backtrace
exit!(1)
end
#Thisisanextensionofthecommonnodedefinition,asitmakesprovisions
#forcustomizingthemasterformoreseamlessinteractionwiththebase
#system
defprovision_master(config,node,attributes)
#Manifestsandmodulesneedtobemountedonthemasterviasharedfolders,
#butthedefault/vagrantmounthaspermissionsandownershipthatconflicts
#withthemasterandpe-puppet.Wemountthesefoldersseparatelywith
#loosenedpermissions.
config.vm.share_folder'manifests','/manifests','./manifests',:extra=>'fmode=644,dmode=755,fmask=022,dmask=022'
config.vm.share_folder'modules','/modules','./modules', :extra=>'fmode=644,dmode=755,fmask=022,dmask=022'
#Updatepuppet.conftoaddthemanifestdirdirectivetopointtothe
#/manifestsmount,ifthedirectiveisn'talreadypresent.
node.vm.provision:shelldo|shell|
shell.inline=<<-EOT
sed-i'
2{
/manifest/!i
manifestdir=/manifests
}
'/etc/puppetlabs/puppet/puppet.conf
EOT
end
#Updatepuppet.conftoaddthemodulepathdirectivetopointtothe
#/modulemount,ifithasn'talreadybeenset.
node.vm.provision:shelldo|shell|
shell.inline=<<-EOT
sed-i'
/modulepath/{
/vagrant/!s,$,:/modules,
}
'/etc/puppetlabs/puppet/puppet.conf
EOT
end
#Rewritetheoldesite.ppconfigsinceit'snotused,andwarnpeople
#aboutthis.
node.vm.provision:shelldo|shell|
shell.inline=%{echo"#/etc/puppetlabs/puppet/manifestsisnotused;see/manifests.">/etc/puppetlabs/puppet/manifests/site.pp}
end
#BoostRAMforthemastersothatactivemqdoesn'tasplode
node.vm.customize(["modifyvm",:id,"--memory","1024"])
#Enableautosigningonthemaster
node.vm.provision:shelldo|shell|
shell.inline=%{echo'*'>/etc/puppetlabs/puppet/autosign.conf}
end
end
#Addsthevagrantconfigurationtweaks
defconfigure_node(config,node,attributes)
node.vm.box=attributes["boxname"]
#Applyallspecifiedportforwards
attributes["forwards"].eachdo|(src,dest)|
node.vm.forward_portsrc,dest
endifattributes["forwards"]#<--Iamamonster
#Addinoptionalper-nodeconfiguration
node.vm.box_url=attributes["boxurl"]ifattributes["boxurl"]
node.vm.network:hostonly,attributes["address"]ifattributes["address"]
node.vm.boot_mode=attributes[:gui]ifattributes[:gui]
end
#Provideprovisioningdetailsforthisnode
defprovision_node(config,node,attributes)
#HackinfauxDNS
#PuppetenterpriserequiressomethingresemblingfunctioningDNStobe
#installedcorrectly
attributes["hosts_entries"].eachdo|entry|
node.vm.provision:shelldo|shell|
shell.inline=%{grep"#{entry}"/etc/hosts||echo"#{entry}">>/etc/hosts}
end
end
#Setthemachinehostname
node.vm.provision:shelldo|shell|
shell.inline=%{hostname#{attributes["name"]}}
end
node.vm.provision:shelldo|shell|
shell.inline=%{domainnamepuppetlabs.test}
end
end
definstall_pe(config,node,attributes)
#Customizetheanswersfileforeachnode
node.vm.provision:shelldo|shell|
shell.inline=%{sed-e's/%%CERTNAME%%/#{attributes["name"]}/'</vagrant/answers/#{attributes["role"]}.txt>/tmp/answers.txt}
end
#IfthePEversionis0.0.0,bypasspuppetinstallation.
#Thiscouldeasilymakelaterprovisioningfail.
ifattributes['pe']['version'].match%r[^0.0]
warn"RequestedPEversionwassetto#{attributes['pe']['version']};willnotautomaticallyinstallPE"
return
end
#Assembletheinstallercommand
fragments=[]
fragments<<"2>&1"
fragments<<attributes['pe']['installer']['executable']
fragments<<'-a/tmp/answers.txt'
fragments<<attributes['pe']['installer']['args'].join('')
installer_cmd=fragments.join('').gsub(':version',attributes['pe']['version'])
#InstallPuppetEnterpriseifithasn'tbeeninstalledyet.Ifthemachine
#isrebootedthenthisprovisioningstepwillbeskipped.
node.vm.provision:shelldo|shell|
shell.inline=<<-EOT
if![-f/opt/pe_version];then
#{installer_cmd}
fi
EOT
end
end
Vagrant::Config.rundo|config|
#GeneratealistofnodeswithstaticIPaddresses
hosts_entries=nodes.select{|h|h["address"]}.map{|h|%{#{h["address"]}#{h["name"]}}}
#TweakeachhostforPuppetEnterprise,andtheninstallPEitself.
nodes.eachdo|attributes|
config.vm.defineattributes["name"]do|node|
attributes["hosts_entries"]=hosts_entries
configure_node(config,node,attributes)
provision_node(config,node,attributes)
install_pe(config,node,attributes)
ifattributes["role"].match/master/
provision_master(config,node,attributes)
end
end
end
end

26. Vagrant and networking
No default internal networking
Flaky DHCP
DNS resolution

27. Vagrant and networking
Solutions!
Run your own infrastructure!
DHCPD
BIND

28. The reality
Can’t support every configuration
Shouldn’t support every configuration
Batteries not included

29. Introducing Oscar

30. Oscar in a nutshell
Set of Vagrant plugins
Built for mutable environments

31. An overview
vagrant-hosts
vagrant-auto_network
vagrant-pe_build
vagrant-config_builder
oscar

32. Vagrant hosts
DNS record types:
starting with A: A, AAAA, AFSDB, APL
starting with C: CAA, CERT, CNAME
starting with D: DHCID, DLV, DNAME, DNSKEY, DS
…
We need: name -> ip address

33. Vagrant hosts
Inside of a transitory environment
Query private network addresses
-> /etc/hosts
Go do something you care about
Or manage BIND on $platform

34. Vagrant Auto-network
Have to add extra interfaces
f$&k it. STATIC IP ADDRESSES FOR ALL
config.vm.network :private_network, :auto_network => true

35. Vagrant PE Build
PE configuration optimized for Vagrant
Download installers on demand

36. Vagrant config builder
Configuration as data

37. Before config builder
Vagrantfile
config.vm.define :puppetmaster do |box|
flavor = :centos_6
set_box box, S3_BOXES[:centos_64_nocm]
# NOTE: Hostonly _may_ mean no internets if this adapter gets found first!!!
# Check /etc/resolv.conf !
box.vm.network :hostonly, "192.168.23.20"
# NOTE: Share folders, such as Git checkouts of the Puppet source code
share_puppet_source box
box.vm.provision :shell, :inline => BOOTSTRAPS[flavor]
provision_box box, 'server.pp'
end

38. After config builder
roles.yaml
---
roles:
puppetmaster:
private_networks:
- {auto_network: true}
synced_folders:
- {host_path: ~/Source/puppetlabs, guest_path: /puppetlabs}
provisioners:
- {type: hosts}
- {type: pe_bootstrap, role: master}
puppetagent:
private_networks:
- {auto_network: true}
provisioners:
- {type: hosts}
- {type: pe_bootstrap}

39. After config builder
vms.yaml
---
vms:
-
name: master
box: centos_64_nocom
roles: puppetmaster
-
name: agent
box: debian_64_nocm
roles: puppetagent

40. Pick your data source
YAML
JSON
XML
interpretive dance

41. What does Oscar do?
Dependencies!
Everything is a standalone plugin
Mix and match

42. What does Oscar do?
Templates and defaults
Sane defaults to get you started

43. PE stack in a box
Configure your development environment like production
Develop your modules in complete isolation
Simulate app deployments before going live
Pre-production in a box!
Stable Puppet environment

44. What Oscar gets you
All the perks of Vagrant
Minimal user setup
Complex config made easy

45. What Oscar gets you
Goal - from zero to PE in vagrant up

46. Who uses it?
Commercial Support
Open Source Support
Sales Engineering
People… and stuff…

47. Demo time!
Basic spin up
Sales eng env

48. Questions?

49. Try it yourself
Code on Github
Under active development
contributions accepted!

50. Thanks
Tom Linkin!
Chris Barker!
Charlie Sharpsteen!
Hunter Haugen!
Adam Crews!
The fabulous Puppet and Vagrant community!
Exclamation points!
Many others!

51. Credits
Layout by Nanoc
Presentation by Reveal.js
Consciousness by caffeine