"Safer Puppet" in 4 quick demos (Beginner) - Thomas Limoncelli, Stack Exchange

1. Tom Limoncelli, SRE
Safer Puppet in
4 quick demos
@YesThatTom
EverythingSysadmin.com

3. Synonyms
• Source Code Repository:
• Source Code Control System, Revision Control
System, etc.
• Ex: Git, Mercurial, Perforce, Subversion, etc.
• CI System:
• Continuous Integration Automation
• Ex: TeamCity TravisCI, Jenkins, etc
3

4. Automation Anxiety
4

5. What if... ?
• ...I make a change that messes everything up?
• ...I create a system that only I can manage?
5

6. We're only human
• I'm human.
• I make mistakes.
• I forget.
6

7. Git Server
1 2
3 4

8. Part 1: Git
Using a Source Code Control
System
8

9. Larry
Mary
Sherry
Kerri
Barry
HarryJerry
9

10. Git
Client
Git Server
$::kernel {
'Linux': {
class { "$
{title}::linux":

11. Demo
11

12. Benefits:
• History of what changed
• Log of who made changes
• Revert some or all changes
19

13. Don’t fear Git
• It is powerful (confusing)
• But you only need to learn the basics
• Tip: Use what your developers use.

14. Git Server
1 2
3 4

15. Git Server
1 2
3 4

16. Part 2: CI
Continuous Integration
23

17. Before a deploy I should...
• Run syntax checker ("puppet parser validate")
• Run unit tests ("rake spec")
• I forget.
24

18. Better would be...
• I make a change.
• Something automatically runs the tests.
• IF the tests pass, copy the files to Puppet Master
25

19. Continuous Integration and
Delivery
• Continuous: like a math function (no gaps)
• Integration: Integrate the latest changes
• Delivery: Deliver results to beta or production
26

20. CI Software
• TeamCity
• TravisCI
• Jenkins/Hudson
• Bamboo
• BuildBot
• BuildForge
• CircleCI
• CruiseControl
• GoCI
• Tinderbox
27

21. Demo
28

22. StackOverflow testing
• Started small:
• Syntax checks
• "puppet parser validate"
• Added new tests over time
• JSON syntax.
39

23. Fixes…
• Automation that has to be run from Larry's account.
• Automation that anyone can use, only if you can
remember the command.
• People skip steps (ignorance or laziness)
40

24. Works for more than Puppet
• Software package builds
• Infrastructure descriptions
• DNS zone file updates
• Anything you store as source code
41

25. Git Server
1 2
3 4

26. Git Server
1 2
3 4

27. Part 3: Vagrant
Personal Sandboxes
44

28. When to use
• Big changes / Starting new modules
• Need fast iterations
• Experiments that intentionally break things
45

29. Sandbox
• A place where I can experiment, try things out.
• A place where I can be free to fail in private.
(“Place” == Machines)
46

30. Option A:
Buy hardware
• Duplicate the environment in hardware.
• Time to buy, rack mount, install, configure
• 2 months
• Just kidding. Your boss won't approve this.
47

31. Option B:
Use your VMware Cluster
• Might require approvals, allocations, etc.
• Not so separate from the rest of the company.
• Re-creating from scratch may take a while.
48

32. Option C:
VMs on your laptop
• Do it now.
• Tear down and rebuild any time you want.
49

33. Vagrant
• Open source project
that makes it easy to set
up test environments.
• Describes the VM at a
high level "IP address,
memory size, etc"
• Does the right thing
whether you use
VirtualBox, VMware,
Hyper-V, Docker.
50

34. Demo: Use Vagrant to set up a
private Puppet Master and Client
• master1: CentOS7, 172.28.19.20
• client1: CentOS7, 172.28.19.22
51

35. 52

36. 53

37. 54

38. 55

39. 56

40. 57

41. Vagrant + VirtualBox
• VirtualBox: Developed by Sun Microsystems, acquired by Oracle
• Vagrant and VirtualBox are both open source, available for free
• Vagrant also supports:
• VirtualBox
• VMware
• Docker
• Hyper-V
58

42. Puppet directory linked from
laptop to VM
• Use my favorite editor on my laptop…
• …changes appear on the Puppet master’s /etc/
puppet
• On my laptop: ~/gitwork/puppet
• On master1: /etc/puppet/environments/vagrant
59

43. 60

44. Demo
61

45. Other benefits
• VMs can be rebuilt from scratch in 5-10 minutes.
• Create many Vagrantfiles, one for each
environment
62

46. Git Server
1 2
3 4

47. Git Server
1 2
3 4

48. Part 4: Environments
Test against production
(without affecting production)
65

49. test == production?
• In theory, you can make your Vagrant environment
look “just like production”
• But what about slight differences?
• Hostnames, network IPs, unpatched OS, etc?
66

50. Puppet Environments
• Lets you use different Puppet source on different
machines.
• Git branch “production” (production servers)
• Git branch “dev” (dev servers)
67

51. Or, use temporary branches!
• Create a git branch with specific changes.
• Use that branch (environment) on specific
machines
68

52. Enabling Technologies
• Branches are cheap in “git”
• R10K makes each “branch” its own “environment”
• StackExchange built a home-brew solution

53. Demo
70

54. Benefits
• Reveals actual changes in real environment
• Useful for refactoring (no changes expected)
74

55. Summary
Source Code Control:
• Track changes
• Undo/revert changes
• Ex: Git
Continuous Integration:
• Automate processes
related to source code
changes
• Add more testing over
time
• Ex: TravisCI, TeamCity,
Jenkins, etc.
Sandboxes:
• Your own development
environment, free.
• http://vagrantup.com
Environments:
• Test and run code from a
Git branch
• Ex: R10K or homegrown
75