It's crucial to make sure everyone's local environment is identical if you're a part of geographically distributed team and your project is composed of many services. It's also quite unlikely that all cogs in your app will spin self-sufficiently. Search engines, various APIs, authentication endpoints and HTTP accelerators - all of these services are standard these days. Your app is a part of a much wider picture. This complexity forces you not only to broaden your mind during design phase, but it also affects the way your software is deployed. All the moving parts should be available on every single environment - including the one that works on your machine. Having in mind aforementioned complexity double click on a ZIP file is a poor man's solution. Duct taping things will end badly. And if anything can go wrong, it will, so better be prepared. Our goal was clear - portable, versionable and predictable environment everyone can run. Throughout our journey with Vagrant, Packer and Chef we learned a lot of things: - how box over-optimization can lead to poor user experience - why it's important to make first "vagrant up" as quick as possible - not everything should be dockerized - Vagrant box is a 'virtual' release artifact and needs the same care your app receives - successful Packer build is often half of the story - why Consul deployment in a Vagrant box is not as trivial as it may sound - how to optimize long Chef runs and provide more granular approach to your dev team - it's surprisingly easy to get from "I don't want to run any VMs on my laptop!" to "Hope you have Vagrant for that"

1. How to stay sane during your
Vagrant journey
Jakub Wądołowski

2. • Focus goes primarily towards staging and production
• Local environments are often underrated and left behind
• Engineers need their own playgrounds
Environment hierarchy
https://flic.kr/p/daxeMY

3. Why should I care about local
environments?

4. Everyone runs the same
https://flic.kr/p/aiBHiz

5. Fast onboarding
https://flic.kr/p/dMP2zA

6. Break things!
https://flic.kr/p/c7a8iE

7. Experiment!
https://flic.kr/p/pZg3vQ

8. Start over any time you need it
https://flic.kr/p/8Mw9gq

9. Before Vagrant

10. • ZIP files everywhere
• Build it once and ZIP it!
• …
• xyz-aem-author.zip
• xyz-aem-author (copy).zip
• xyz-aem-author_20140112.zip
• xyz-aem-publish_backup.zip
• xyz-aem-publish_recovery_061213.zip
• john’s-aem-author (backup).zip
• 1307120312-aem (copy).zip
Back in the days…
https://flic.kr/p/nD6HMe

11. • The overall complexity increased
• Much more moving parts
• Whatever Works™ doesn’t scale anymore
• New technologies emerged
Things have changed
https://flic.kr/p/EQbNYd

12. • Isolation
• Reliability
• Completeness
• Ability to upgrade
• Decent UX
The right balance
https://flic.kr/p/6cKwNy

13. • Makes very little sense due to underlying architecture
• Introduces differences between dev and prod
• Vast majority of the team runs Windows
Why not Docker?
https://flic.kr/p/9Y5SVV

14. Vagrant beginnings

15. VAGRANTFILE_API_VERSION = '2'
Vagrant.require_version '>= 1.5.0'
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.hostname = 'project-vagrant'
config.vm.box = 'opscode-centos-6.6'
config.vm.box_url = 'http://whatever.s3.amazonaws.com/opscode-centos-6.6.box'
config.vm.network :private_network, ip: '192.168.123.123'
config.vm.network 'forwarded_port', guest: 6002, host: 4502
config.vm.network 'forwarded_port', guest: 26002, host: 24502
config.vm.provider 'virtualbox' do |v|
v.memory = 3072
v.cpus = 2
end
config.omnibus.chef_version = '11.16.4'
config.berkshelf.enabled = true
config.vm.provision :chef_solo do |chef|
chef.json = {}
chef.run_list = [
'recipe[cognifide-base::default]',
'recipe[project-webapp::author_base]',
'recipe[project-webapp::author_app]',
'recipe[project-httpd::default]'
]
end
end

16. • Pros
• full application stack
• provisioned the same way as all other
environments (Chef)
• all in one box
• managed as Chef cookbook
Nothing extraordinary, huh?
https://flic.kr/p/dwvcSo
• Cons
• based on completely clean box
• first vagrant up takes over 2 hours
• everyone waits for the same process
to complete
• poor user experience

17. Custom boxes

18. • Veewee was first (released in 2011)
• Packer came into play in 2013
• Both tools require something that underpins the build process
Build your own box
https://flic.kr/p/cMBbTu

19. • In-house tool that goes through 3 phases
• prepare
• build (Packer)
• release
• Requires physical machine
Build process
https://flic.kr/p/GQopsq

21. {
"builders": [
{
"type": "virtualbox-ovf",
"vm_name": "{{user `image_name`}}"
}
],
"post-processors": [
{
"output": "{{user `packer_build_dir`}}/{{.Provider}}/{{user `image_name`}}-{{user `image_version`}}.box",
"type": "vagrant"
}
],
"provisioners": [
{
"chef_environment": "app-packer",
"run_list": [
"recipe[xyz-base::hostname]",
"recipe[xyz-base::lvm]"
],
"validation_key_path": "{{user `chef_validation_key_path`}}"
},
{
"scripts": [
"scripts/aem/remove_tmp.sh",
"scripts/generic/minimize.sh"
],
"type": "shell"
}
],
"variables": {
"chef_validation_key_path": "{{env `CHEF_VALIDATION_KEY_PATH`}}",
"description": "xyz full-stack box",
"image_name": "xyz-full-stack",
"image_version": "0.9.1",
"packer_build_dir": "{{env `PACKER_BUILD_DIR`}}",
}
}

22. • Build takes 1-3 hours (varies by project)
• Packer destroys everything on error
• Say hello to github.com/mitchellh/packer/issues/409
• Extremely frustrating at the very beginning
• Even more frustrating when it happens during box export
Take your time…
https://flic.kr/p/e88Ce2

23. • Set TMP variable to custom location
• Use environment variables for configuration
• Save Chef environment to file upon successful build
• Create tags in your repository
• Reduce box size, but…
• Great examples: github.com/chef/bento
Build tuning
https://flic.kr/p/9hTC27

24. • Why one VM is better than many?
• Consider incremental boxes
• Don’t repeat yourself (start from scratch)
Even more tuning
https://flic.kr/p/kmYSYT

25. • HashiCorp Atlas
• Build your internal Vagrant Cloud Atlas
• 3rd party services, i.e. Artifactory
Box distribution
https://flic.kr/p/5EBCiH

26. VAGRANTFILE_API_VERSION = '2'
Vagrant.require_version '>= 1.5.0'
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.hostname = 'project-vagrant'
config.vm.box = 'project-full-stack'
config.vm.box_url = 'https://boxes.example.com/project-full-stack/'
config.vm.box_version = '0.6.2'
config.vm.box_check_update = true
config.vm.network :private_network, ip: '192.168.123.123'
config.vm.network 'forwarded_port', guest: 6002, host: 4502
config.vm.network 'forwarded_port', guest: 26002, host: 24502
config.vm.provider 'virtualbox' do |v|
v.memory = 3072
v.cpus = 2
end
config.omnibus.chef_version = '11.16.4'
config.berkshelf.enabled = true
config.vm.provision :chef_zero do |chef|
chef.json = {}
chef.run_list = [
'recipe[cognifide-base::default]',
'recipe[project-webapp::author_base]',
'recipe[project-webapp::author_app]',
'recipe[project-httpd::default]'
]
end
end

27. • vagrant up provides complete environment
• Don’t force vagrant provision on first run
• All services are in place
Final product
https://flic.kr/p/gs9NMG

28. On-going work with Vagrant

29. • Preserve habits and routines
• Compile/build/install flow stays the same
• No config changes is required
• Feels like a first-class environment
• Easy to upgrade
Provisioning rules
https://flic.kr/p/csrifU

30. • git pull && berks update && vagrant provision
• git pull && berks update && vagrant provision
• git pull &&
Keep it simple
https://flic.kr/p/iYHqv2

31. Featureful, but slow and heavy
https://flic.kr/p/9nHAfM

32. Better UX , much faster
https://flic.kr/p/nA2Tf1

33. • Automate repetitive tasks
• Reduce CLI complexity
• Eliminate confusion related to Berkshelf
• 15 minutes to deploy a single config change?!
• 8/789 resources updated in 04 minutes 41 seconds?!
Time to rake
https://flic.kr/p/aEaKWh

34. Vagrant.require_version '~> 1.8.1'
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.hostname = 'project-vagrant'
config.vm.box = 'project-full-stack'
config.vm.box_url = 'https://boxes.example.com/project-full-stack/'
config.vm.box_version = '1.2.1'
config.vm.box_check_update = true
config.vm.provider 'virtualbox' do |v|
v.memory = ENV['VAGRANT_MEM'] || 6144
v.cpus = ENV['VAGRANT_CPU'] || 2
v.linked_clone = true
end
config.berkshelf.enabled = false if Vagrant.has_plugin?('vagrant-berkshelf')
config.vm.provision :chef_solo do |chef|
chef.node_name = 'project-vagrant'
chef.cookbooks_path = 'cookbooks'
run_lists = {}
run_lists['empty'] = []
run_lists['httpd'] = [ 'recipe[project-httpd::default]' ]
run_lists['tomcat_baseline'] = [ 'recipe[project-tomcat::default]' ]
run_lists['tomcat_full'] = [
'recipe[project-tomcat::default]',
'recipe[project-tomcat::app]'
]
# Fallback to empty run list if there's no .run_list file
rl = File.read('.run_list') rescue 'empty'
chef.run_list = run_lists[rl]
File.delete('.run_list') rescue nil # Once read this file is redundant
end
end

35. services = [
{ 'httpd' => 'Provision Apache configuration' },
{ 'tomcat_baseline' => 'Provision Tomcat' },
{ 'tomcat_full' => 'Provision Tomcat and install apps' },
]
namespace 'berkshelf' do
file 'Berksfile.lock' => 'Berksfile' do
sh 'berks update || berks install'
Rake::Task['berkshelf:vendor'].execute
end
task :vendor do
sh 'berks vendor cookbooks'
end
end
namespace 'vagrant' do
task :up do
sh 'vagrant up --no-provision --no-color'
end
task :provision => 'Berksfile.lock' do
sh 'vagrant provision --no-color'
end
end
namespace 'provision' do
services.each do |s|
desc "#{s.values.first}"
task :"#{s.keys.first}" do
File.open('.run_list', 'w') { |f| f.write(s.keys.first) }
Rake::Task['vagrant:provision'].invoke
end
end
end

36. desc 'Install all required Vagrant plugins'
task :init do
sh 'vagrant plugin install vagrant-hostmanager'
end
desc 'Remove old Vagrant boxes to reclaim disk space'
task :clean do
vf = File.read('Vagrantfile')
box_name = vf[/config.vm.box = '([^']+)'/,1]
box_version = vf[/config.vm.box_version = '([^']+)'/,1]
old_versions = []
%x(vagrant box list).lines.grep(/#{box_name}/).each do |l|
# Line format: BOX_NAME (PROVIDER, VERSION)
m = l.match(/(?:[^ ]+) +((?:[^,]+), (?<version>.+))/)
old_versions.push(m['version']) if m['version'] != box_version
end
old_versions.each do |v|
sh "vagrant box remove #{box_name} --box-version #{v}"
end
end
desc 'Start Vagrant and skip provisioning part'
task :up => 'vagrant:up'
task :provision => 'vagrant:provision'
desc 'Refresh all cookbook dependecies'
task :refresh do
sh 'berks update'
Rake::Task['berkshelf:vendor'].invoke
end

37. The reward
https://flic.kr/p/do51TU

38. $ rake -T
rake clean # Remove old Vagrant boxes to reclaim disk space
rake refresh # Refresh all cookbook dependecies
rake init # Install all required Vagrant plugins
rake provision:httpd # Provision Apache configuration
rake provision:tomcat_baseline # Provision Tomcat
rake provision:tomcat_full # Provision Tomcat and install all apps
rake up # Start Vagrant and skip provisioning part
$ rake provision:httpd
vagrant provision --no-color
==> default: Running provisioner: chef_solo...
==> default: Detected Chef (latest) is already installed
==> default: Generating chef JSON and uploading...
==> default: Running chef-solo...
==> default: Compiling Cookbooks…
...
==> default: [2016-06-10T08:35:02+02:00] INFO: Chef Run complete in 9.854542964 seconds
==> default: [2016-06-10T08:35:02+02:00] INFO: Skipping removal of unused files from the cache
==> default:
==> default: Running handlers:
==> default: [2016-06-10T08:35:02+02:00] INFO: Running report handlers
==> default: Running handlers complete
==> default: [2016-06-10T08:35:02+02:00] INFO: Report handlers complete
==> default: Chef Client finished, 0/34 resources updated in 11 seconds

39. • Does it make any sense?
• Consul + dnsmasq
• Update nameservers in /etc/resolv.conf
• Prevent DNS changes via /etc/dhcp/dhclient-enter-hooks
• Works most of the time
Service discovery and Vagrant
https://flic.kr/p/83mv7r

40. make_resolv_conf(){
:
}
# /etc/dhcp/dhclient-enter-hooks: line 1: make_resolv_conf(){
# /etc/dhcp/dhclient-enter-hooks: line 1: syntax error near unexpected token {

41. • EOL characters (.gitattributes)
• Chef Solo vs Chef Zero
• https://github.com/chef/chef/issues/4980
• Less plugins equals faster processing
• Useful helpers
• Vagrant.require_version
• Vagrant.has_plugin?(‘plugin')
• Vagrant::VERSION =~ /^1.8/
Vagrant tips
https://flic.kr/p/nzMBtw

42. https://flic.kr/p/tt2yB7