"Dev to Delivery with Puppet, Vagrant and AWS" by Sam Bashton of Bashton Ltd. at Puppet Camp London 2013. Find the video here: http://puppetlabs.com/community/puppet-camp

1. DEV TO DELIVERY WITH
PUPPET, VAGRANT AND AWS
SAM BASHTON, BASHTON LTD

3. ABOUT ME
Linux guy since Slackware, floppy disks and root + boot
Using Puppet since 2007
Run a company in Manchester, North West England
We provide outsourced ops for other companies

4. TOOLS FOR THE DAY

5. WHAT IS THE POINT OF THIS
TALK?

7. WHAT WE HAVE
Dev
Integration
QA
Stage
Live

8. WHICH ENVIRONMENTS ARE
MANAGED BY PUPPET?
Dev
Integration
QA
Stage
Live

9. WHAT WE'RE AFTER
Confidence that everything will work correctly in
production
Consistency between environments

10. OPS AND DEVS COOPERATING
Previously:
Devs built stuff
Later, Ops came and built production infrastructure
This caused many IT problems
The solution?

11. OPSVELOPMENT

12. OPS AND DEVS WORKING
TOGETHER
Ops need to be involved in development planning process
Puppet modules and manifests should be selected/built as
part of the development process

13. DEVELOP ON PUPPET
PROVISIONED
ENVIRONMENTS
As early as possible, all dev should be done on systems
built from Puppet
Puppet manifests get tested as part of the development
process

14. VAGRANT
Builds virtual machines, optionally from Puppet manifests
Makes it easy to spin up short-lived dev instances
Quick to get working
Avoid ops being a blocker for dev

15. A WORKFLOW
Development happens on Vagrant VM(s)
Deployment to all shared environments happens via
Jenkins

16. PUPPET CONFIG
There should be only one set of Puppet
manifests/modules
Tested deployed and merged through software test
environments

17. ONE SET OF MANIFESTS,
MANY ENVIRONMENTS
Different environments need different config
Resource locations
Settings

18. DEALING WITH DIFFERENT
ENVIRONMENTS
Hiera
Removes the need for ugly if/else blocks
Put anything that differers by environment in a separate
file
Can encrypt with hiera-gpg if data sensitive

19. HIERA.YAML
:irrh:
heacy
-%evrnet
{niomn}
-cmo
omn

20. VAGRANTFILE
VgatcniueVGATIEAIVRIN d |ofg
arn.ofgr(ARNFL_P_ESO) o cni|
cni.mbx="ets4lc
ofgv.o
cno6-x"
cni.mhsnm ="uptofeape
ofgv.otae
ppecn-xml"
cni.mpoiin:uptd |upt
ofgv.rvso ppe o ppe|
ppe.aiet_ah
uptmnfsspt
="uptmnfss
ppe/aiet"
ppe.aietfl
uptmnfs_ie
="iep"
st.p
ppe.ouept
uptmdl_ah
="uptmdls
ppe/oue"
ppe.ir_ofgpt ='uptheaym'
uptheacni_ah
ppe/ir.al
ppe.pin = [-evrnet,"oadv]
uptotos > "-niomn" lcle"
ed
n
ed
n

21. TO DEVELOP:
Start of the day, dev runs v g a t u and gets the latest
arn p
environment
Code/objects sit in a shared vagrant volume
End of the day, or when new Puppet manifests/modules
are available, v g a t d s r yis run
arn eto

22. VAGRANT PROVISIONERS
Avoid VirtualBox wherever possible
Slow, prone to taking down host machine
On Linux, vagrant-lxc is speedy
VMWare Fusion for non-free fruit-based Unix

23. VAGRANT AND AWS
Use Vagrant to bring up machines in AWS using
v g a t a splugin
arn-w
Makes it easy to share work in progress
Means VirtualBox doesn't crash your laptop
Has cost implications

24. QA/STAGING
ENVIRONMENTS IN AWS
Merge to appropriate branch in git
Jenkins takes over

25. ADVANTAGE OF AWS
Great thing about AWS - we don't need to run our test
environments all the time
Have the environments only when you need them

26. TESTING VS LIVE
Use the money saved to build better environments
Minimise differences between testing and live
In particular, test on environments with relevant HA as
early as possible

29. SPEEDING UP THE PROCESS
Some resources, in particular DBs can be slow to provision
(30 mins plus)
Could just run 24/7
One approach: pilot light provisioning

32. PILOT LIGHT PROVISIONING
Tiers built using autoscaling groups
Minimum instance count is 0
Jenkins sets desired capacity appropriately on deploy
Reset to 0 via a recurring scheduled operation on ASG
and/or Jenkins job

33. CONCLUSIONS
Infrastructure development should run in parallel to
software dev
This means devs + ops must co-operate
Minimise differences from production at all stages
If a dev can't see the problem in their environment,
you're much more likely to get woken up by it

34. QUESTIONS? COMMENTS?
Sam Bashton
sam@bashton.com
Twitter: @bashtoni
(Psst.. http://www.bashton.com/jobs/ )

35. REFERENCES, LINKS
Vagrant
vagrant-lxc
hiera-gpg
Masterless Puppet + AWS