Buildout is a great tool for creating repeatable environments. But what happens when things change and you are left with a broken buildout? This talk will focus on creating a buildout that can survive the test of time. The talk will cover how to keep your packages pinned so that releases don't cause you headaches. We will also discuss how to set up your own package server in order to make sure that you always have the packages that you need.

1. Clayton Parker, Senior Web Developer
Buildout for the Future
PLONE CONFERENCE 2010
Thursday, October 28, 2010

2. PLONE CONFERENCE 2010Who Am I?
• claytron
• Python dev since 2003
• Plone Core Committer
• Foundation Member
Thursday, October 28, 2010

3. PLONE CONFERENCE 2010What Will We
Learn?
• Pinning
• Indexes
• Buildout Tricks
Thursday, October 28, 2010

4. PLONE CONFERENCE 2010Problems with
Buildout
• PyPi
• Lost/deleted packages
• Version conflicts
Thursday, October 28, 2010

5. PLONE CONFERENCE 2010PinningVersions
• [versions]
• Local config
• Extended URLs
Thursday, October 28, 2010

6. PLONE CONFERENCE 2010Versions
[buildout]
# tell buildout to use the [versions] part
versions = versions
[versions]
my.package = 1.0
some.other.package = 2.0
Thursday, October 28, 2010

7. PLONE CONFERENCE 2010PloneVersions
[buildout]
extends = http://dist.plone.org/release/4.0.1/versions.cfg
Thursday, October 28, 2010

8. PLONE CONFERENCE 2010Plone versions.cfg
[buildout]
extends = http://download.zope.org/Zope2/index/2.12.11/versions.cfg
[versions]
# Buildout infrastructure
plone.recipe.zope2instance = 4.0.4
plone.recipe.zeoserver = 1.1.1
...
# External dependencies
Markdown = 2.0.3
PIL = 1.1.6
...
# Plone release
Plone= 4.0.1
...
Thursday, October 28, 2010

9. PLONE CONFERENCE 2010Zope versions.cfg
[buildout]
versions = versions
[versions]
Zope2 = 2.12.11
...
Thursday, October 28, 2010

10. PLONE CONFERENCE 2010Our versions.cfg
## buildout.cfg
[buildout]
extends =
http://dist.plone.org/release/4.0.1/versions.cfg
versions.cfg
## versions.cfg
[versions]
my.package = 1.0
some.other.package = 2.0
Thursday, October 28, 2010

11. PLONE CONFERENCE 2010GettingVersions
$ bin/buildout -v
...
Installing instance
We have the distribution that satisfies 'Plone==4.0.1'.
We have the distribution that satisfies 'plone.app.caching==1.0b1'.
We have the distribution that satisfies 'plone.app.ldap==1.2.3'.
We have the distribution that satisfies 'plone.reload==1.5'.
We have the distribution that satisfies 'zope.testrecorder==0.4'.
We have the distribution that satisfies 'Products.PDBDebugMode==1.1'.
We have the distribution that satisfies 'Products.PrintingMailHost==0.7'.
We have the distribution that satisfies 'Products.DocFinderTab==1.0.4'.
We have the distribution that satisfies 'Products.signalstack==1.0rc2'.
We have the distribution that satisfies 'Products.PTProfiler==1.4'.
We have the distribution that satisfies 'Products.DCWorkflowGraph==0.4'.
We have the distribution that satisfies 'ipdb==0.2'.
We have the distribution that satisfies
'plone.recipe.zope2instance==4.0.4'.
Getting required 'Zope2==2.12.11'
We have the distribution that satisfies 'Zope2==2.12.11'.
Thursday, October 28, 2010

12. PLONE CONFERENCE 2010Dump Picked
Versions
[buildout]
extensions = buildout.dumppickedversions
Thursday, October 28, 2010

13. PLONE CONFERENCE 2010Dump Picked
Versions
$ bin/buildout -v
...
*************** PICKED VERSIONS ****************
[versions]
PILwoTk = 1.1.6.4
Products.DCWorkflowGraph = 0.4
Products.DocFinderTab = 1.0.4
Products.PDBDebugMode = 1.1
Products.PTProfiler = 1.4
Products.PrintingMailHost = 0.7
Products.signalstack = 1.0rc2
ipdb = 0.2
plone.app.caching = 1.0b1
plone.app.ldap = 1.2.3
zope.testrecorder = 0.4
*************** /PICKED VERSIONS ***************
Thursday, October 28, 2010

14. PLONE CONFERENCE 2010Extended URLs
[buildout]
extends =
# Plone's set of version dependencies
http://dist.plone.org/release/4.0.1/versions.cfg
# specific versions for plone.app.caching package
http://good-py.appspot.com/release/plone.app.caching/1.0b1
# our version overrides and add-on packages
versions.cfg
Thursday, October 28, 2010

15. PLONE CONFERENCE 2010Public Indexes
• PyPi
• Mirrors
• Plone dist
• http://dist.plone.org/
• Company dist
• http://dist.company.com/public/
Thursday, October 28, 2010

16. PLONE CONFERENCE 2010Find Links
[buildout]
find-links = http://dist.plone.org/release/4.0.1/
[buildout]
plone-version = 4.0.1
extends = http://dist.plone.org/release/${:plone-version}/versions.cfg
find-links = http://dist.plone.org/release/${:plone-version}/
Thursday, October 28, 2010

17. PLONE CONFERENCE 2010Private Indexes
• Customer eggs
• Customer archives
• Not ready for pypi
• Branches / trunk
Thursday, October 28, 2010

18. PLONE CONFERENCE 2010Solutions
• Apache / nginx
• Plone Software Center
• basketweaver
• haufe.eggserver
• ClueReleaseManager
Thursday, October 28, 2010

19. PLONE CONFERENCE 2010Apache
• Use directory listings
• Basic auth for protection
• mod_ldap for integration
Thursday, October 28, 2010

20. PLONE CONFERENCE 2010Config
<Location /private/myproject>
AuthType Basic
AuthName "myproject"
# hook up to ldap or a password file
# ...
</Location>
Thursday, October 28, 2010

21. PLONE CONFERENCE 2010lovely.buildouthttp
• Access private egg repos
• Also tied into download recipes
myproject,https://dist.example.com/private/myproject,username,password
Thursday, October 28, 2010

22. PLONE CONFERENCE 2010Releasing Eggs
• jarn.mkrelease
• zest.releaser
Thursday, October 28, 2010

23. PLONE CONFERENCE 2010~/.pypirc
[distutils]
index-servers =
pypi
plone.org
[pypi]
username: username
password: password
[plone.org]
repository: http://plone.org/products
username: username
password: password
Thursday, October 28, 2010

24. PLONE CONFERENCE 2010~/.mkrelease
[defaults]
distbase =
distdefault = public
[aliases]
public = dist.company.com:/var/dist/public
myproject = dist.company.com:/var/dist/private/myproject
world =
public
pypi
plone-world =
world
plone.org
Thursday, October 28, 2010

25. PLONE CONFERENCE 2010mkrelease
$ mkrelease -d myproject
$ mkrelease -d public
$ mkrelease -d plone-world
Thursday, October 28, 2010

26. PLONE CONFERENCE 2010Buildout Profiles
• Debugging / local dev
• Production
Thursday, October 28, 2010

27. PLONE CONFERENCE 2010Profiles
my-buildout
"## buildout.cfg
"## parts
"## profiles
$ "## base.cfg
$ "## debug.cfg
$ "## local.cfg
$ "## prod.cfg
$ &## versions.cfg
&## var
Thursday, October 28, 2010

28. PLONE CONFERENCE 2010Annotate
• Check config
• Debug issues
• Verify settings
Thursday, October 28, 2010

29. PLONE CONFERENCE 2010Annotate
$ bin/buildout annotate
$ bin/buildout -c profiles/prod.cfg annotate
Thursday, October 28, 2010

30. Check out
sixfeetup.com/demos
Thursday, October 28, 2010