A talk about methods and tools to automate deployment of Plone sites. With a few steps an environment is prepared for a new Plone site on a test, staging or production layer. These steps take a couple of minutes, doing this manually took around one hour. We use Puppet to prepare our hosts/clusters to get an environment to deploy to. Fabric is used to deploy Plone on this environment and to extend the webserver configuration under the hood. These complementary techniques provide a complete solution to get a working Plone site, including rollbacks. Presentation by: Pawel Lewicki and Kim Chee Leong

1. Plone deployment
made easy
Pawel Lewicki
Kim Chee Leong
Goldmund, Wyldebeast & Wunderliebe
{ lewicki, leong } @gw20e.com

2. Outline
● From manual to automatic deployment
● How Puppet and Fabric are used for
deployment
● Deployment demo in screencast
● Puppet server orchestration
● Fabric in detail

3. From manual to
automatic deployment
our path to automatic deployment

4. Typical Plone release
@GWW
● Mark eggs/modules using git tag
● Update mr developer sources with tag
● SSH to server
● Clone new buildout in releases directory
● Bootstrap buildout and execute
● Switch previous release to new release
● If on a cluster set-up, rinse and repeat

5. Plone release @GWW
using Fabric
● All previous steps in one Fabric script
● Allows ‘dummy-proof’ releases
Run:
fab make_tag
fab deploy
fab switch
And Bob is your uncle!

6. xkcd - automation
http://xkcd.com/1319/

7. Why automatic
deployment?
● Avoid repetitive tasks
● Error-proof
● Immune to typo’s
● Consistent environments
● Time efficient

8. Why automatic
deployment?
● Avoid repetitive tasks
● Error-proof
● Immune to typo’s
● Consistent environments
● Time efficient
nothing* should be done manually
on the server

9. How Puppet and Fabric
are used for deployment

10. Initial server setup with
Puppet
● Puppet is used for server orchestration
● Puppet prepares the servers
● When Puppet is done, Fabric is used to
deploy buildouts

11. Puppet diagram

12. Buildout deployment with
Fabric
● Uses tst, acc and prd layers
● Knows configuration for buildout
● Deploys and runs buildouts
● Does several other tasks

13. Fabric diagram

14. Deployment screencast
using Puppet and Fabric

15. Puppet server
orchestration
paving the way for Fabric

16. Example Puppet config
node 'ploneconf.puppet' {
package { 'apache2': ensure => installed }
appie::app { "ploneconfapp":
envs => {
prd => { uid => 3011 },
acc => { uid => 3012 },
tst => { uid => 3013 },
},
webserver => 'apache2',
accountinfo => $gw20e::user_accounts,
}
class { 'ssh':
server_options => {
'PasswordAuthentication' => 'no',
'PermitRootLogin' => 'no',
},
}
}
1 / n

17. Puppet tasks
● User accounts for TAP environments:
o app-mysite-prd in /opt/APPS/mysite/prd
o app-mysite-acc in /opt/APPS/mysite/acc
o app-mysite-tst in /opt/APPS/mysite/tst
● Webserver config is mapped to each
application user
$ cat /etc/apache2/sites-enabled/zzz-app-mysite-prd
Include /opt/APPS/mysite/prd/sites-enabled/
● SSH Keys of developers are synced
● Buildout cache is enabled

18. Fabric in detail
using gww.buildout and gww.buildout-fabric

19. Plone Fabric features
● Automatic deployment
● Push buildout config using jinja templates
● Extended TAP-layers
● Switching between releases
● But also; supports git branches, virtual host
for webserver, copying zodb to local
buildout.

20. Release switch using
Fabric
1. Update current symlink
rm -f ~/current
ln -s ~/releases/20141028 ~/current
1. Start supervisor daemon in new buildout
~/releases/20141028/bin/supervisord
1. For each service in supervisor:
a. Stop service in old buildout
b. Start service in new buildout
2. Shutdown supervisor in old buildout
~/releases/20130101/bin/supervisorctl shutdown

21. ● Fabric is controlled from the local buildout:
./bin/fab -l
Available commands:
deploy Create new buildout in release dir
switch Switch supervisor to latest buildout
test Test if the connection is working
● Deploying to test environment:
./bin/fab deploy:layer=tst,branch=new-feature

22. Example deployment config (1/2)
prdfrontend = dict(
hosts=prd['hosts'],
buildout='releases/frontend',
varnish={'port': 48083},
haproxy={'port': 48082, 'instances': prd['instances'], },
webserver='apache2',
sitename='ploneconfapp-prd.puppet',
site_id=prd['site_id'],
)
prdbackend = dict(
hosts=prd['hosts'],
buildout='releases/backend',
zeo=dict(base='/data1/APPS/ploneconfapp/prd', **prd['zeo']),
)

23. Example Deployment config (2/2)
prd = dict(
hosts=['app-ploneconfapp-prd@ploneconf.puppet',],
buildout=_datestamped('releases/%Y-%m-%d'),
current_link='current',
auto_switch=False,
modules=_modules,
third_party_modules=_third_party_modules,
zeo={
'ip': ‘ploneconf.puppet’, 'port': 48081,
},
instances={
'ports': {'instance0': 8080, 'instance1': 8081},
'ipaddresses': _servers,
},
site_id=_site_id,
credentials={'username': 'admin', 'password': 'secret', },
remote_configs={'clockusers': 'clockuser.cfg'},
sentry={
'dsn': 'https://usr:passwd@sentry.gw20e.com/29',
'level': 'ERROR',
},
)

24. Example jinja template / buildout
[supervisor]
programs +=
...
{% if varnish %}
60 varnish ${buildout:directory}/bin/varnish true
{% endif %}
{% if varnish %}
[varnish]
port = {{ varnish.port }}
{% endif %}

25. Example jinja template / apache
{% if varnish %}
{% set port = varnish.port %}
{% elif haproxy %}
{% set port = haproxy.port %}
{% else %}
{% set port = instances.ports.instance0 %}
{% endif %}
<VirtualHost *:80>
ServerName {{ sitename }}
RewriteEngine on
ProxyPass / http://localhost:{{ port }}/VirtualHostBase/http/{{ sitename }}:80/
ProxyPassReverse / http://localhost:{{ port }}/../http/{{ sitename }}:80/
</VirtualHost>

26. Github repositories
● https://github.com/search?q=ploneconf2014_gww
● Puppet module:
o http://git.io/2NLMtg (shortened url)
o https://github.com/Goldmund-Wyldebeast-Wunderliebe/puppet-appie
● Fabric module:
o http://git.io/kVdsnw (shortened url)
o https://github.com/Goldmund-Wyldebeast-
Wunderliebe/gww.buildout-fabric
● GWW Plone buildout
o http://git.io/PzdjqQ (shortened url)
o https://github.com/Goldmund-Wyldebeast-
Wunderliebe/gww.buildout

27. Thank you!
Goldmund, Wyldebeast & Wunderliebe
{ lewicki, leong } @gw20e.com