Paver For PyWorks 2008

1. Smoothing Software Project Scripting Kevin Dangoor BlueSkyOnMars.com php|works PyWorks 2008

2. Python is not compiled (actually, it is, but that’s not important right now)

3. Python projects certainly do not need • Redistributable files to move around • Deployment to other machines • Interaction with source control systems • Documentation that’s built from the source code • Documentation that’s built from a format other than its final display format

5. If only there was some scripting language that we already knew.

6. Waitasec. Isn’t Python a scripting language?

7. Project-related scripts need... • Command line argument handling • Configuration • Often work with files • Sometimes need to use distutils/ setuptools, but wish they could do a little more • Need to work with other common tools (Sphinx, svn, virtualenv)

8. Command-line argument handling bin/start-server --port 8675309

9. Configuration [messages] greeting=Hello [thing1] who=World message=${messages.greeting}, ${who}

10. Lots of working with files import os if not os.path.exists(“foo”): os.mkdir(“foo”) if not os.path.exists(os.path.join(“foo, “bar”)): open(os.path.join(“foo”, “bar”), “w”).write(“Hi”)

11. Working with distutils/setuptools #!/bin/sh sphinx-build (blah blah blah) python setup.py sdist upload

12. Other common tools sphinx-build ... subprocess.Popen(“svn info”... # and do something with the output virtualenv.create_bootstrap_script(“# more code”)

13. Do you really want separate scripts for everything? • /usr/local/bin/git /usr/local/bin/git-merge-resolve • /usr/local/bin/git-fast-import /usr/local/bin/git-runstatus • /usr/local/bin/git-add /usr/local/bin/git-merge-stupid • /usr/local/bin/git-fetch /usr/local/bin/git-send-email • /usr/local/bin/git-add--interactive /usr/local/bin/git-merge-subtree • /usr/local/bin/git-fetch--tool /usr/local/bin/git-send-pack • /usr/local/bin/git-am /usr/local/bin/git-merge-tree • /usr/local/bin/git-fetch-pack /usr/local/bin/git-sh-setup • /usr/local/bin/git-annotate /usr/local/bin/git-mergetool • /usr/local/bin/git-filter-branch /usr/local/bin/git-shell • /usr/local/bin/git-apply /usr/local/bin/git-mktag • /usr/local/bin/git-fmt-merge-msg /usr/local/bin/git-shortlog • /usr/local/bin/git-archimport /usr/local/bin/git-mktree • /usr/local/bin/git-for-each-ref /usr/local/bin/git-show • /usr/local/bin/git-archive /usr/local/bin/git-mv • /usr/local/bin/git-format-patch /usr/local/bin/git-show-branch • /usr/local/bin/git-bisect /usr/local/bin/git-name-rev • /usr/local/bin/git-fsck /usr/local/bin/git-show-index • /usr/local/bin/git-blame /usr/local/bin/git-pack-objects • /usr/local/bin/git-fsck-objects /usr/local/bin/git-show-ref • /usr/local/bin/git-branch /usr/local/bin/git-pack-redundant • /usr/local/bin/git-gc /usr/local/bin/git-ssh-fetch • /usr/local/bin/git-bundle /usr/local/bin/git-pack-refs • /usr/local/bin/git-get-tar-commit-id /usr/local/bin/git-ssh-pull • /usr/local/bin/git-cat-file /usr/local/bin/git-parse-remote • /usr/local/bin/git-grep /usr/local/bin/git-ssh-push • /usr/local/bin/git-check-attr /usr/local/bin/git-patch-id • /usr/local/bin/git-gui /usr/local/bin/git-ssh-upload • /usr/local/bin/git-check-ref-format /usr/local/bin/git-peek-remote • /usr/local/bin/git-hash-object /usr/local/bin/git-stash • /usr/local/bin/git-checkout /usr/local/bin/git-prune • /usr/local/bin/git-http-fetch /usr/local/bin/git-status • /usr/local/bin/git-checkout-index /usr/local/bin/git-prune-packed • /usr/local/bin/git-imap-send /usr/local/bin/git-stripspace • /usr/local/bin/git-cherry /usr/local/bin/git-pull • /usr/local/bin/git-index-pack /usr/local/bin/git-submodule • /usr/local/bin/git-cherry-pick /usr/local/bin/git-push • /usr/local/bin/git-init /usr/local/bin/git-svn • /usr/local/bin/git-citool /usr/local/bin/git-quiltimport • /usr/local/bin/git-init-db /usr/local/bin/git-svnimport • /usr/local/bin/git-clean /usr/local/bin/git-read-tree • /usr/local/bin/git-instaweb /usr/local/bin/git-symbolic-ref • /usr/local/bin/git-clone /usr/local/bin/git-rebase • /usr/local/bin/git-local-fetch /usr/local/bin/git-tag • /usr/local/bin/git-commit /usr/local/bin/git-rebase-- • /usr/local/bin/git-log /usr/local/bin/git-tar-tree interactive • /usr/local/bin/git-lost-found /usr/local/bin/git-unpack-file • /usr/local/bin/git-commit-tree /usr/local/bin/git-receive-pack • /usr/local/bin/git-ls-files /usr/local/bin/git-unpack-objects • /usr/local/bin/git-config /usr/local/bin/git-reflog • /usr/local/bin/git-ls-remote /usr/local/bin/git-update-index • /usr/local/bin/git-convert-objects /usr/local/bin/git-relink • /usr/local/bin/git-ls-tree /usr/local/bin/git-update-ref • /usr/local/bin/git-count-objects /usr/local/bin/git-remote • /usr/local/bin/git-mailinfo /usr/local/bin/git-update-server-info • /usr/local/bin/git-cvsexportcommit /usr/local/bin/git-repack • /usr/local/bin/git-mailsplit /usr/local/bin/git-upload-archive • /usr/local/bin/git-cvsimport /usr/local/bin/git-repo-config • /usr/local/bin/git-merge /usr/local/bin/git-upload-pack • /usr/local/bin/git-cvsserver /usr/local/bin/git-request-pull • /usr/local/bin/git-merge-base /usr/local/bin/git-var • /usr/local/bin/git-daemon /usr/local/bin/git-rerere • /usr/local/bin/git-merge-file /usr/local/bin/git-verify-pack • /usr/local/bin/git-describe /usr/local/bin/git-reset • /usr/local/bin/git-merge-index /usr/local/bin/git-verify-tag • /usr/local/bin/git-diff /usr/local/bin/git-rev-list • /usr/local/bin/git-merge-octopus /usr/local/bin/git-whatchanged • /usr/local/bin/git-diff-files /usr/local/bin/git-rev-parse • /usr/local/bin/git-merge-one-file /usr/local/bin/git-write-tree • /usr/local/bin/git-diff-index /usr/local/bin/git-revert • /usr/local/bin/git-merge-ours /usr/local/bin/gitk • /usr/local/bin/git-diff-tree /usr/local/bin/git-rm • /usr/local/bin/git-merge-recursive

15. easy_install Paver

16. paver paverdocs

17. pavement.py

18. Why Python build files? • You already know Python • The language rules are well-defined • The language rules are well- documented • Python is powerful, so you’ll never be left hanging or need an escape hatch

19. Pavements are almost standard Python from paver.defaults import *

21. Configuration options( setup = setup_meta, minilib=Bunch( extra_files=['doctools', 'virtual'] ), sphinx=Bunch( builddir=quot;buildquot;, sourcedir=quot;sourcequot; ), virtualenv=Bunch( packages_to_install=[quot;nosequot;, quot;sphinxquot;, quot;docutilsquot;, quot;virtualenvquot;], install_paver=False, script_name='bootstrap.py', paver_command_line=None ) )

28. Dynamic config values >>> from paver.defaults import * >>> import time >>> options(current=lambda: time.time()) >>> options.current 1216726815.0027969

29. Namespace searching >>> options( ... setup=Bunch(version=quot;1.0quot;), ... sphinx=Bunch(builddir=quot;docbuildquot;) ... ) >>> options.version '1.0'

30. Namespace searching (continued) >>> options( ... setup=Bunch(version=quot;1.0quot;), ... sphinx=Bunch(builddir=quot;docbuildquot;) ... ) >>> options.order('sphinx') >>> options.version Traceback (most recent call last): File quot;<stdin>quot;, line 1, in <module> File quot;/Users/admin/projects/paver/paver/runtime.pyquot;, line 31, in __getattr__ raise AttributeError(name) AttributeError: version

33. Configuration is still standard Python • You can treat options like a normal, nested dictionary • The only unusual thing would be that callables are called.

35. Tasks @task def clean(): quot;quot;quot;Cleans up this paver directory. Removes the virtualenv traces and the build directory.quot;quot;quot; pass

40. paver help $ paver help ---> help Paver 0.8.1 Usage: paver [global options] [option.name=value] task [task options] [task...] Run 'paver help [section]' to see the following sections of info: options global command line options setup available distutils/setuptools tasks tasks all tasks that have been imported by your pavement 'paver help taskname' will display details for a task. Tasks defined in your pavement: bootstrap - Build a virtualenv bootstrap for developing

45. paver help tasks Tasks defined in and imported by your pavement: bootstrap - Build a virtualenv bootstrap for developing paver clean - Cleans up this paver directory cog - Runs the cog code generator against the files matching your specification commit - Removes the generated code from the docs and then commits to bzr deploy - Copy the Paver website up doc_clean - Clean (delete) the built docs generate_setup - Generates a setup help - Displays the list of commands and the details html - Build Paver's documentation and install it into paver/docs minilib - Create a Paver mini library that contains enough for a simple

46. paver help <taskname> Details for minilib: Create a Paver mini library that contains enough for a simple pavement.py to be installed using a generated setup.py. This is a good temporary measure until more people have deployed paver. The output file is 'paver-minilib.zip' in the current directory. Options: extra_files list of other paver modules to include (don't include the .py extension)

47. @needs @task @needs(quot;uncogquot;) def commit(): quot;quot;quot;Removes the generated code from the docs and then commits to bzr.quot;quot;quot; pass

50. call_task call_task(‘commit’)

51. @cmdopts @task @cmdopts([(quot;username=quot;, quot;uquot;, quot;Username for remote serverquot;), (quot;server=quot;, quot;squot;, quot;Server to deploy toquot;)]) def deploy(): quot;quot;quot;Copy the Paver website up.quot;quot;quot; pass

52. paver <taskname> $ paver generate_setup minilib ---> generate_setup Write setup.py ---> minilib Generate paver-minilib.zip

53. Paver and Distutils/ Setuptools

54. “Using the Distutils is quite simple, both for module developers and for users/ administrators installing third-party modules.” – Distributing Python Modules, section 1.1

55. python setup.py install

56. Paver extends Distutils

57. paver install

58. setup.py example from distutils.core import setup setup(name='foo', version='1.0', py_modules=['foo'], )

63. Upgrading to Paver options( setup = Bunch(name='foo', version='1.0', py_modules=['foo'], ) )

64. Keeping it simple for users $ paver generate_setup minilib ---> generate_setup Write setup.py ---> minilib Generate paver-minilib.zip

65. Generated setup.py import os if os.path.exists(quot;paver-minilib.zipquot;): import sys sys.path.insert(0, quot;paver-minilib.zipquot;) import paver.command paver.command.main()

66. The Paver Standard Library (is actually newer and less musty)

67. The Paver Standard Library • Runtime helpers (paver.runtime) • Distutils integration (paver.setuputils) • File handling (paver.path) • Documentation tools (paver.doctools) • Subversion (paver.svn) • SSH (paver.ssh) • Virtualenv (paver.virtual) • Miscellaneous Tasks (paver.misctasks)

68. paver.runtime # display text if verbose is set debug(“Hi there. Feeling chatty today?”) # display text if quiet is not set info(“Glad we don’t have to keep quiet”) # display text regardless of setting error(“HA! I’M SHOUTING AND YOU CAN’T STOP ME!”)

75. paver.runtime (continued) # run a command, as long as dry-run is off # capture the output into myval myval = sh(“cat /tmp/foo”, capture=True) # run a function (delete_all(‘/’) to be exact). # if dry-run is set, then # just print the message instead dry(“Delete everything”, delete_all, ‘/’)

80. paver.setuputils options.setup.package_data = setuputils.find_package_data(quot;paverquot;, package=quot;paverquot;, only_in_packages=False)

81. paver.path • Jason Orendorff’s path.py module (also available separately) • It’s a subclass of string! • Fun use of operator overloading • Lots of great methods • Makes working with files/directories fun and easy

82. paver.path examples p = path(quot;docsquot;) tmpdir = p / quot;tmpquot; tmpdir.mkdir() fn = tmpdir / quot;myfile.txtquot; fn.write_text(quot;Hi there!quot;)

89. paver.doctools # add this to use import paver.doctools

90. paver.doctools – Sphinx paver.doctools.html()¶ Build HTML documentation using Sphinx. This uses the following options in a “sphinx” section of the options. docroot the root under which Sphinx will be working. Default: docs builddir directory under the docroot where the resulting files are put. default: build sourcedir directory under the docroot for the source files default: (empty string) paver.doctools.doc_clean()¶ Clean (delete) the built docs. Specifically, this deletes the build directory under the docroot. See the html task for the options list.

91. paver.doctools – Cog & SectionedFile • Solutions to common problems of creating high-quality docs • Ideally, your code samples will be in convenient runnable code files and have unit tests. • But you also want nice, minimal code samples in your text as you’re writing.

92. paver.doctools sample code # mysample.py # [[[section mysample]]] def sample_func(): print quot;To sample, or not to sample?quot; # [[[endsection]]]

93. paver.doctools in docs And then when you want to print the sample string, you just call:: # [[[cog include(“code/mysample.py”, “mysample”)]]] # [[[end]]]

94. paver.doctools in docs (2) And then when you want to print the sample string, you just call:: # [[[cog include(“code/mysample.py”, “mysample”)]]] def sample_func(): print quot;To sample, or not to sample?quot; # [[[end]]]

95. paver.doctools uncog before commit @task @needs(quot;uncogquot;) def commit(): quot;quot;quot;Removes the generated code from the docs and then commits to bzr.quot;quot;quot; sh(quot;bzr commitquot;)

96. paver.svn paver.svn.checkout(url, dest, revision='') Checks out the speciﬁed URL to the given destination. paver.svn.checkup(url, dest, revision='') Does a checkout or update, depending on whether the destination exists and is up to date (if a revision is passed in). Returns true if a checkout or update was performed. False otherwise. paver.svn.info(path='') Retrieves the svn info for the path and returns a dictionary of the values. Names are normalized to lower case with spaces converted to underscores. paver.svn.update(path='', revision='') Run an svn update on the given path. Example at: https://projects.sitepen.com/toolbox/svn/trunk/pavement.py

97. paver.ssh paver.ssh.scp(source, dest) Copy the source file to the destination.

98. paver.virtual paver.virtual.bootstrap() Creates a virtualenv bootstrap script. The script will create a bootstrap script that populates a virtualenv in the current directory. The environment will have paver, the packages of your choosing and will run the paver command of your choice. This task looks in the virtualenv options for: script_name name of the generated script packages_to_install packages to install with easy_install. The version of paver that you are using is included automatically. This should be a list of strings. paver_command_line run this paver command line after installation (just the command line arguments, not the paver command itself).

99. paver.misctasks paver.misctasks.generate_setup() paver.misctasks.help() paver.misctasks.minilib() Options: extra_ﬁles list of other paver modules to include (donʼt include the .py extension) paver.misctasks.paverdocs()

100. Pulling it all together • write functions with the @task decorator. • Just plain old Python with lots of conveniences • Paver’s pavement.py is a good example • Even better, take a look at “Getting Started with Paver” http://bit.ly/starting_paver

101. Remember this? #!/bin/sh sphinx-build (blah blah blah) python setup.py sdist upload

102. Paver needs to do that too @task @needs(['html', quot;minilibquot;, quot;generate_setupquot;, “setuptools.command.sdist”]) def sdist(): quot;quot;quot;Builds the documentation and the tarball.quot;quot;quot; pass

103. But wait, it does more! @task @needs(['cog', 'paver.doctools.html']) def html(): quot;quot;quot;Build Paver's documentation and install it into paver/ docsquot;quot;quot; builtdocs = path(quot;docsquot;) / options.sphinx.builddir / quot;htmlquot; destdir = path(quot;paverquot;) / quot;docsquot; destdir.rmtree() builtdocs.move(destdir)

104. Changes coming to Paver • The little bit of magic goes away • Support for CloudControl • Ability to run sub-builds easily • Virtualenv/pyinstall management • zc.buildout recipe running

105. Credits • Thanks to SitePen for supporting Paver • Bumpy road http://flickr.com/photos/ilya/442034/sizes/o/ • Smooth road http://flickr.com/photos/nicholas_t/317528561/ • Options (Anna’s hairdo) http://flickr.com/photos/alangley/2136034468/ • Tasks http://flickr.com/photos/mpwillis/470557535/

106. http:// www.blueskyonmars. com/projects/ paver/

Paver For PyWorks 2008

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (18)

Destaque

Destaque (16)

Semelhante a Paver For PyWorks 2008

Semelhante a Paver For PyWorks 2008 (20)

Último

Último (20)

Paver For PyWorks 2008