My presentation from PyCodeConf 2011.
I talk about:
- how the Python open-source community is a meritocracy
- how the best way to grow your ecosystem within the Python community is to make it easy to create third-party packages
- community building, mentorship, and diversity of ideas
1. Audrey Roy
@audreyr
Amazing things in
open source
3rd-party Python package ecosystems
Audrey Roy
@audreyr
2. Audrey Roy
@audreyr
About me
• Python volunteer work: Django Packages/
OpenComparison co-BDFL, PyLadies co-organizer
• Entrepreneur: co-founder of Whitespace Jobs
• Django consultant: Cartwheel Web & Revsys
• Fiancee of Daniel Greenfeld (@pydanny)
3. Audrey Roy
@audreyr
Overview
• Python community is a meritocracy
• Python ecosystem patterns
• 3rd-party packages: Quantity & quality
• Community-building
4. Audrey Roy
@audreyr
Meritocracy
If your work has merit, people use it
5. Audrey Roy
@audreyr
We are a meritocracy
• Python open-source community is a meritocracy
• Anyone can build anything
• Anyone can start a user group
• Anyone can be a leader
• ...as long as they put in the work
6. Audrey Roy
@audreyr
No permission needed
• Just implement or experiment with what you
want
• The more useful or helpful your work is, the
more you succeed
• (A few exceptions)
7. Audrey Roy
@audreyr
It’s fun
• Rewarding to see others using your work
8. Audrey Roy
Meritocracies are about what
@audreyr
you actually do
• Call to action: Build whatever you desire/need
• Don’t worry about politics. Just be nice
• If you don’t build X, no one will
• You can solve your own problems
9. Audrey Roy
@audreyr
Who’s in charge?
If you’re doing the coding, you’re in charge*. So
make the decisions that you want.
*when it’s your own project or 3rd-party Python package
10. Audrey Roy
Early decisions for
@audreyr
djangopackages.com
• Forced to make quick decisions during Django Dash
• All packages are added manually, using:
• package name
• PyPI URL
• repo URL
11. Audrey Roy
@audreyr
Decisions, decisions
• No automatic package-finding spiders, by design.
• Was it a good decision?
• Who knows, doesn’t matter
• Currently 900+ packages
12. Audrey Roy
@audreyr
Decisions, decisions
• Sometimes you just have to pick something
• As opposed to asking a mailing list and having X
get debated for 2 weeks
• Mailing lists can be useful, but be careful
13. Audrey Roy
@audreyr
The Future is in Your Hands
(or is it?)
• Future of your Python project: In your hands/in the
hands of your core dev team
• or...
• Future of your Python project: Decided by other
people on a mailing list
14. Audrey Roy
@audreyr
Your Gut Instinct Is Often Right
• Fun fact: many of the grid items/descriptions on
Django Packages come from “throwaway” fixtures
that I created in a rush
• Don’t like them? You have edit permissions ;)
15. Audrey Roy
@audreyr
Ecosystem Patterns
Add-ons, plugins, contrib, core size, etc.
What makes a Python project or framework grow?
16. Audrey Roy
Why I Use Mostly Django
@audreyr
These Days
• Many 3rd-party packages
• Candy store
• Bucket of Legos
• Not having to write everything myself is fun
17. Audrey Roy
Why I Use Mostly Django
@audreyr
These Days
• (also, I’m just used to it)
18. Audrey Roy
@audreyr
Django Core vs. Apps
• Many, many batteries included
• Gives you one obvious way to do things
• Third-party apps: “Django packages”
• Good and bad
19. Audrey Roy
Django’s Reusable App
@audreyr
Structure
• What an app looks like
houses/
__init__.py
models.py
views.py
urls.py
tests.py
templates/houses/
house_list.html
house_detail.html
20. Audrey Roy
@audreyr
Installing Django Apps
• How to install an app
# settings.py
INSTALLED_APPS += (‘houses’)
YOUR_CUSTOM_SETTING = ‘your value’
# urls.py
url(r’^houses/’, include(‘houses.urls’)),
21. Audrey Roy
@audreyr
Django’s Ecosystem Over Time
• More and more new innovations implemented as
3rd-party packages.
• The best innovations get added to core
22. Audrey Roy
@audreyr
Observation
• Ecosystems grow fastest when end-users can write
packages
• Look at jQuery plugins
• Look at PyPI
• Look at CPAN
23. Audrey Roy
@audreyr
Pyramid Core vs. Add-Ons
• Smaller core, more add-ons
• Anyone can write add-ons
• Some add-ons are “officially endorsed”
• Still young, but potential for rapid growth
24. Audrey Roy
@audreyr
Pyramid’s Ecosystem Over Time
• Past: Pylons, Repoze.BFG, Turbogears
• Present: Core-like functionality implemented as
add-ons.
• Few but growing # of add-ons
• Future: Lots of add-ons. Old add-ons easily
deprecated for new replacement add-ons.
25. Audrey Roy
Checklist: What 3rd-Party
@audreyr
Package Devs Need
• “Best practices” doc on how to write 3rd party
packages
• Well-defined, easy-to-understand spec
• Sample code (as much as possible)
• Active community
26. Audrey Roy
Checklist: What Every Micro-
@audreyr
Community Needs
• Mailing list for announcements/questions
• IRC channel
• Docs, tutorials, sample projects
• Spec or “Best practices” doc for package authors.
• Add-on, plugin, or other 3rd-party package system
• Actively-updated 3rd-party package catalog
27. Audrey Roy
@audreyr
Too many options?
Dealing with a (not always) happy problem
28. Audrey Roy
@audreyr
The One True Way?
• The Zen of Python says: “ There should be one--
and preferably only one --obvious way to do it.”
• This is about Python language constructs
• Not applicable to 3rd-party packages
• e.g. okay to have multiple web frameworks
29. Audrey Roy
@audreyr
Too Much Clutter
Can’t find the right package?
• What if 2 packages have the same purpose?
• Differences should be documented
• Within the package docs
• In a catalog
• Deprecate bad packages
30. Audrey Roy
@audreyr
Too Much Fragmentation?
• So many Python micro-communities
• Web frameworks with their own user bases
• SciPy, PyGame, wxPython, etc.
31. Audrey Roy
@audreyr
Too Much Fragmentation?
• You can never have too many Python interest
groups
• Diversity of ideas
• More communities == more Python developers
• Share as much code as possible
32. Audrey Roy
@audreyr
3rd Party Package
Quality
Patterns and Anti-Patterns
33. Audrey Roy
What Makes a Python Package
@audreyr
Useful?
• Solving problems elegantly, clearly, efficiently
• Does one thing well
• Usability: thorough docs, pip install from PyPI
• Reliability: tests, support
34. Audrey Roy
@audreyr
Package Anti-Patterns Are Viral
• Don’t underestimate the influence of your patterns
• When there is no best practice, people find the
anti-pattern code samples
35. Audrey Roy
Package Anti-Pattern:
@audreyr
Not Packaging
• Releasing snippets instead of packages
• Heavy reliance on copying and pasting snippets
• project-to-project
• from old blog posts
• snippet directories
36. Audrey Roy
Package Anti-Pattern:
@audreyr
Not Packaging
• But there are exceptions to this!
• Don’t over-engineer to be pluggable
37. Audrey Roy
Package Anti-Pattern:
@audreyr
Not Packaging
• Tricks that you have to be “in the know” to
understand
38. Audrey Roy
Package Anti-Pattern:
@audreyr
Too Much Functionality
• Kitchen-sink base platforms
• Utility, do-everything packages
• Duct-tape packages to correct a large variety of
unrelated problems
39. Audrey Roy
@audreyr
Package Pattern: Be Pythonic
• Why do we love Python?
• Elegance
• Ease of use
• Explicitness, clarity
• Simplicity
40. Audrey Roy
@audreyr
Community-building
Groups, leaders, and diversity of ideas
41. Audrey Roy
@audreyr
Mentorship
• Today’s new users are tomorrow’s contributors &
leaders
• Mentorship groups: Positive encouragement
• PyLadies
• Python Core Mentorship
• (need more like this)
42. Audrey Roy
@audreyr
Diversity of Ideas
• Look at schedules & slides from PyCons & PUGs
around the world
• Ideas differ country-to-country
• Same goes for other types of diversity besides
geographic
• LA PyLadies is very different from SoCal Python
Interest Group & LA Django
44. Audrey Roy
@audreyr
Summary
• Build what you want
• Encourage a community of 3rd-party package
developers
• Be helpful
45. Audrey Roy
@audreyr
Thank you
• Twitter: @audreyr
• Find me in IRC - #pyladies, others
• By the way:
Editor's Notes
\n
\n
Without 3rd party packages, your language, project, or framework relies on the core team. No ecosystem. The core team can only develop so fast. Why do I like Django so much? Because of how many packages are available for me to use for my Django projects.\n
\n
\n
You want to play with the GIL? With PyPy? Just download the code and experiment. It’s okay by default\n
\n
\n
\n
Balance between curation and automation: you have to cut and paste URLs manually, but you don’t have to enter too much data into the system\n
\n
\n
\n
\n
\n
I’m primarily a Django web developer these days.\nThere are so many Django packages that anything feels possible.\n
I know Django well enough to implement what I want to build with it. \n
\n
\n
\n
\n
\n
\n
\n
\n
If you’re a leader of a Python sub-community, think of this as your checklist.\nIf you’re a member of one and want to become a leader, fill in the gaps for your sub-community.\n\nPlugin systems, even if very basic, encourage 3rd-party package development.\njQuery plugins, PyPI\ndjango.contrib has growing objections\n
\n
\n
You can help your ecosystem by documenting the differences\nTired of maintaining a package? Don’t be afraid to say “Use X instead”\n\n
The Zen of Python says “There should be one-- and preferably only one --obvious way to do it.”\nThat should not be applied to 3rd-party packages\nIf that were the case, there would be only 1 web framework\nDiversity of ideas and packages is what draws new users to Python. So many modules to experiment with!\n
If there’s a generic Python way of implementing something, do it that way\n
\n
Has a purpose, without going overboard\nUsability: docs explain how to integrate it with your code.  Easy to download the standard pip requirements file way.\nReliability: tests, ongoing leadership, maintenance, and support\n\n
\n
Snippets are good when they supplement a package. Sample code illustrating how something works is good.\nRelying on snippets as your crutches to build too many things is bad.\n
Sometimes a package overcomplicates things or just doesn’t make sense. For example, Leah Culver showed some good counter-examples yesterday with Backbone.js. Don’t over-engineer and force a simple problem to fit into an overly complex Python package.\n
A lot of the time documentation solves the problem, but sometimes this is also a symptom of a gap: an opportunity for someone to create a helpful 3rd-party package\n
You see this a lot in the HTML/CSS world, where the CSS reset has grown out of control into do-everything resets.  There are some big-name CSS projects out there that do this, and I’m afraid to name names. Thankfully Python is better about it.\nHard to swap out pieces (in many cases)\nLook at jQuery UI for a nice solution to this: check off boxes and download your own custom package\n
Your end-users should have as good an experience with using your 3rd-party Python package as they do with using the Python programming language itself.\n
\n
It’s hard to put yourself out there. Mentorship groups offer more positive encouragement, and less cynicism/criticism.\n
e.g. sometimes it feels like the only people who care about having a WSGI 2 spec are from outside the US. \nPyCon Australia: Fearless across-the-board comparison of topics, lots of focus on core Python & Python 3\nKiwi PyCon: More Twisted users than I’ve ever met, huge emphasis on testing, e.g. automated unit test generation with Pythoscope\nRuPy in Poland\nLA PyLadies: lots of discussions about Django front-end development, Django and JavaScript, AJAX techniques, joint meetup with node.js user group, plans for a Python-JS conference next year. \n