In the Python community we are taught from the outset of learning the language that the Zen of Python serves as a guide for how we should construct our codebases and projects. Rather than go into the zen-like meanings of each statement, this talk will explore how individual koans are implemented via detailed displays of sophisticated code examples.
Boost PC performance: How more available memory can improve productivity
An Extreme Talk about the Zen of Python
1. An Extreme Talk about
the Zen of Python
Daniel Greenfeld
PyCon Poland
2012
Daniel Greenfeld
pydanny.com / @pydanny
2. @pydanny
• Daniel Greenfeld
• Father’s parents were from
Poland circa 1920.
• Mother’s family were from
Poland circa 1903.
Dynow, Poland
Dynow, Poland
to USA
Daniel Greenfeld
pydanny.com / @pydanny
3. @pydanny
• Learned Python at NASA
• Principal at Cartwheel Web
• Member of Python Software
Foundation
• Member of Django Software
Foundation
Daniel Greenfeld
pydanny.com / @pydanny
4. @pydanny
Audrey Roy (Fiancée)
Daniel Greenfeld
pydanny.com / @pydanny
7. The Zen of Python
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
Daniel Greenfeld
pydanny.com / @pydanny
8. Tim Peters
Author of Timsort
Timsort is a hybrid sorting algorithm, derived from merge sort
and insertion sort, designed to perform well on many kinds of
real-world data. It was invented by Tim Peters in 2002 for use in
the Python programming language. The algorithm finds subsets
of the data that are already ordered, and uses the subsets to sort
the data more efficiently. This is done by merging an identified
subset, called a run, with existing runs until certain criteria are
fulfilled. Timsort has been Python's standard sorting algorithm
since version 2.3. It is now also used to sort arrays in Java SE 7,
and on the Android platform.
https://en.wikipedia.org/wiki/Timsort
Daniel Greenfeld
pydanny.com / @pydanny
10. The Opening
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Daniel Greenfeld
pydanny.com / @pydanny
11. super()
Daniel Greenfeld
pydanny.com / @pydanny
12. Circle
import math
class Circle(object): >> Circle(10)
Circle as area
def __init__(self, radius): 314.159265359
self.radius = radius
def area(self): >>> Ring(10, 5)
return self.radius ** 2 *math.pi 235.619449019
def __repr__(self):
return '{0} as area {1}'.format(
self.__class__.__name__, self.area()
)
The super method calls the
class Ring(Circle): parent class, which is Circle
def __init__(self, outer, inner):
super(Ring, self).__init__(outer)
self.inner = inner What if our inheritance
isn’t simple?
def area(self):
outer, inner = self.radius, self.inner
return Circle(outer).area() - Circle(inner).area()
Daniel Greenfeld
pydanny.com / @pydanny
14. The Opening
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Daniel Greenfeld
pydanny.com / @pydanny
15. The Opening
Ambiguity of
super() method Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Who actually Complex is better than complicated.
remembers the Flat is better than nested.
Sparse is better than dense.
super() syntax? Readability counts.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Ambiguity of In the face of ambiguity, refuse the temptation to guess.
super() method
Daniel Greenfeld
pydanny.com / @pydanny
16. Circle II
import math
class Circle(object): >> Circle(10)
Circle as area
def __init__(self, radius): 314.159265359
self.radius = radius
def area(self): >>> Ring2(10, 5)
return self.radius ** 2 *math.pi 235.619449019
def __repr__(self):
return '{0} as area {1}'.format(
self.__class__.__name__, self.area() Explicit
)
Absolutely inheriting
class Ring2(Circle):
__init__ from Circle Simpler
def __init__(self, outer, inner):
Circle.__init__(self, outer) More readable
self.inner = inner
def area(self):
Note: Only do this when
outer, inner = self.radius, self.inner
return Circle(outer).area() -it.
you need Circle(inner).area()
Daniel Greenfeld
pydanny.com / @pydanny
19. Django class based views
• Composition
• Inheritance
• Subclass
• Polymorphism
• Lots of other big words
Daniel Greenfeld
pydanny.com / @pydanny
20. However...
Daniel Greenfeld
pydanny.com / @pydanny
21. Quiz
What is the ancestor chain for
django.views.generic.edit.UpdateView?
Daniel Greenfeld
pydanny.com / @pydanny
22. Answer
The ancestor chain for django.views.generic.edit.UpdateView:
django.views.generic.edit.UpdateView
django.views.generic.detail.SingleObjectTemplateResponseMixin
django.views.generic.base.TemplateResponseMixin
django.views.generic.edit.BaseUpdateView
django.views.generic.edit.ModelFormMixin
django.views.generic.edit.FormMixin
django.views.generic.detail.SingleObjectMixin
django.views.generic.edit.ProcessFormView
django.views.generic.base.View
Daniel Greenfeld
pydanny.com / @pydanny
23. The Opening
Ambiguity of
super method Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Who actually Complex is better than complicated.
remembers the Flat is better than nested.
Sparse is better than dense.
super() syntax? Readability counts.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Ambiguity of In the face of ambiguity, refuse the temptation to guess.
super method
Daniel Greenfeld
pydanny.com / @pydanny
24. Answer
The ancestor chain for django.views.generic.edit.UpdateView:
django.views.generic.edit.UpdateView
django.views.generic.detail.SingleObjectTemplateResponseMixin
django.views.generic.base.TemplateResponseMixin
django.views.generic.edit.BaseUpdateView
django.views.generic.edit.ModelFormMixin
django.views.generic.edit.FormMixin
django.views.generic.detail.SingleObjectMixin
django.views.generic.edit.ProcessFormView
django.views.generic.base.View
Daniel Greenfeld
pydanny.com / @pydanny
25. A form_valid()
implementation
def form_valid(self, form):
verb_form = verb_form_base(self.request.POST)
if verb_form.is_valid():
form.instance.verb_attributes = verb_form.cleaned_data
return super(ActionUpdateView, self).form_valid(form)
Which form_valid()
am I calling?
Daniel Greenfeld
pydanny.com / @pydanny
26. A form_valid()
OMG! implementation
OMG!
class ActionUpdateView(
LoginRequiredMixin, # django-braces
ActionBaseView, # inherits from AuthorizedForProtocolMixin
AuthorizedforProtocolEditMixin, # Checks rights on edit views
VerbBaseView, # Gets one of 200+ verb forms
UpdateView): # django.views.generic.BaseView
def form_valid(self, form):
verb_form = verb_form_base(self.request.POST)
if verb_form.is_valid():
OMG! form.instance.verb_attributes = verb_form.cleaned_data
return super(ActionUpdateView, self).form_valid(form)
Daniel Greenfeld
pydanny.com / @pydanny
27. Ancestor Chain (MRO)
of ActionUpdateView
from actions.views import ActionUpdateView
for x in ActionUpdateView.mro():
print(x)
Print the MRO
MRO = Method Resolution Order
Daniel Greenfeld
pydanny.com / @pydanny
29. Ancestor Chain (MRO)
of ActionUpdateView
from actions.views import ActionUpdateView
for x in [x for x in ActionUpdateView.mro() if hasattr(x, "form_valid")]:
print(x)
Filter the MRO list to only include
classes with a form_valid() nethod
Daniel Greenfeld
pydanny.com / @pydanny
30. Ancestor Chain (MRO)
of ActionUpdateView
Current class super’s chosen
form_valid() ancestor
<class 'actions.views.ActionUpdateView'>
<class 'django.views.generic.edit.UpdateView'>
<class 'django.views.generic.edit.BaseUpdateView'>
<class 'django.views.generic.edit.ModelFormMixin'>
<class 'django.views.generic.edit.FormMixin'>
Daniel Greenfeld
pydanny.com / @pydanny
33. If you’re not careful,
super can cause subtle
inheritance/MRO
problems.
Daniel Greenfeld
pydanny.com / @pydanny
34. The Opening
Ambiguity of
super method Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Who actually Complex is better than complicated. Possibly these
remembers the Flat is better than nested. as well
Sparse is better than dense.
super() syntax? Readability counts.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Ambiguity of In the face of ambiguity, refuse the temptation to guess.
super method
Daniel Greenfeld
pydanny.com / @pydanny
35. Possible mitigations
for this view.
• Hope that anyone else maintaining this
project isn’t going to kill me.
• Convert to a functional view.
• Explore better patterns.
• return UpdateView.form_valid(self, form)
Daniel Greenfeld
pydanny.com / @pydanny
36. Moving on...
Part II
Daniel Greenfeld
pydanny.com / @pydanny
37. Controversy
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Daniel Greenfeld
pydanny.com / @pydanny
38. Controversy: Django
• WSGI (fixed)
• Configuration and installation (working on it)
• Class Based Views (We’re working on it)
• Not Model-View-Controller compliant
Daniel Greenfeld
pydanny.com / @pydanny
39. Django: Not MVC
• Django follows Model-Template-View.
• Is the web appropriate for MVC?
• The Zen of Python doesn’t mention MVC.
• But it’s all moot because...
...what we really care about is...
Daniel Greenfeld
pydanny.com / @pydanny
41. Controversy
Django is pretty good about
following the Zen of Python
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Well... maybe not CBVs...
Daniel Greenfeld
pydanny.com / @pydanny
42. Controversy: Web2py
• Often honors Implicit over Explicit
• Follows its own namespace pattern
Daniel Greenfeld
pydanny.com / @pydanny
43. Web2py code sample
# encoding: utf-8
# Response object magically exists.
https://github.com/mdipierro/evote/blob/master/models/menu.py
# this file is released under public domain and necessary
No import
# you can use without limitations
response.title = 'Voting Service'
response.subtitle = None
## read more at http://dev.w3.org/html5/markup/meta.name.html
response.meta.author = 'Your Name <you@example.com>'
response.meta.description = 'a cool new app'
response.meta.keywords = 'web2py, python, framework'
response.meta.generator = 'Web2py Web Framework'
# snip more content that I cut in the name of brevity
What about namespace pollution? What can I expect in any location?
Daniel Greenfeld
pydanny.com / @pydanny
44. Contention
Web2py violates these 3 koans:
• Explicit is better than implicit
• In the name of ambiguity, refuse the
temptation to guess
• Namespaces are one honking great idea --
let's do more of those!
Daniel Greenfeld
pydanny.com / @pydanny
45. Controversy
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Daniel Greenfeld
pydanny.com / @pydanny
46. Web2py contends:
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Daniel Greenfeld
pydanny.com / @pydanny
47. Web2py contends:
Note: This is my interpretation of Web2py design considerations.
• Implicit behaviors means Web2py is easier for
beginners to learn.
• The Web2py namespace pattern is easy to learn.
• For experienced developers, commonly repeated
imports are boilerplate.
Personal side note: Web2py is very easy to install.
Daniel Greenfeld
pydanny.com / @pydanny
48. Controversy
Django is pretty good about Web2py argues practicality
following the Zen of Python in some very specific places.
Special cases aren’t special enough to break the rules.
Although practicality beats purity.
Web2py will always
Well... maybe not CBVs...
be contentious
Just like Django.
Daniel Greenfeld
pydanny.com / @pydanny
49. Fixing Exceptions to
Exception handling
Part III
Daniel Greenfeld
pydanny.com / @pydanny
51. Django Packages
• Once a day iterates across all packages.
• Updates the metadata from:
• Github:
• Bitbucket
• PyPI
Daniel Greenfeld
pydanny.com / @pydanny
52. Django Packages
Problems
• Sometimes the APIs go down.
• Sometimes the APIs change.
• Sometimes projects get deleted.
• Sometimes the Internets fail
Catch and report exceptions!
Daniel Greenfeld
pydanny.com / @pydanny
53. package_updater.py
...
for package in Package.objects.all():
try:
package.fetch_metadata()
package.fetch_commits()
except socket_error, e:
text += "nFor '%s', threw a socket_error: %s" %
(package.title, e)
continue
# snip lots of other exceptions Um...
except Exception, e:
text += "nFor '%s', General Exception: %s" %
(package.title, e)
continue
# email later
http://bit.ly/Q8v9xk
https://github.com/opencomparison/opencomparison/blob/master/package/management/commands/package_updater.py
Daniel Greenfeld
pydanny.com / @pydanny
54. What I’m doing now
(and it’s wrong)
>>> try:
... a = b
... except Exception as e:
... print(e)
...
name 'b' is not defined
What’s the Where is my
error type?!? stack trace?!?
Daniel Greenfeld
pydanny.com / @pydanny
55. What I want
Traceback
>>> a = b
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'b' is not defined
Error type
Error message
Daniel Greenfeld
pydanny.com / @pydanny
56. Exceptions
My code is
nearly silent
Errors should never pass silently.
Unless explicitly silenced.
I’ve silenced things
for no good reason
Daniel Greenfeld
pydanny.com / @pydanny
57. Getting what I want
>>> class CustomErrorHandler(Exception):
... def __init__(self, error):
... print(error) For this example
... print(type(error)) print == log
...
>>> try:
... a=b Error message
... except Exception as e:
... raise CustomErrorHandler(e)
Traceback ...
name 'b' is not defined No color because
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
it’s a print
__main__.CustomErrorHandler statement
NameError
Error Type
Daniel Greenfeld
pydanny.com / @pydanny
58. PackageUpdaterException
class PackageUpdaterException(Exception):
def __init__(self, error, title):
log_message = "For {title}, {error_type}: {error}".format(
title=title,
error_type=type(error),
error=error
Nice message
)
logging.error(log_message)
logging.exception(error)
Full traceback
for package in Package.objects.all():
try:
try:
package.fetch_metadata() All errors
package.fetch_commits() caught
except Exception, e:
raise PackageUpdaterException(e, package.title)
except PackageUpdaterException:
continue
Loop forward Daniel Greenfeld
pydanny.com / @pydanny
59. Exceptions
My code is
nearly silent
Errors should never pass silently.
Unless explicitly silenced.
I’ve silenced things
for no good reason
Daniel Greenfeld
pydanny.com / @pydanny
60. Cleaner code
Part IV
Daniel Greenfeld
pydanny.com / @pydanny
61. More controversy
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Daniel Greenfeld
pydanny.com / @pydanny
62. More controversy
You try to shoot yourself in the foot, only
to realize there’s no need, since Guido
thoughtfully shot you in the foot years ago.
-- Nick Mathewson, comp.lang.python
http://starship.python.net/~mwh/quotes.html
Daniel Greenfeld
pydanny.com / @pydanny
63. Decorators
def allcaps(string):
@memoize
def allcaps(string):
return string.upper()
> return string.upper()
allcaps = memoize(allcaps)
Decorators are
easy to explain!
Daniel Greenfeld
pydanny.com / @pydanny
64. Decorator Template
def decorator(function_to_decorate):
def wrapper(*args, **kwargs):
Wrapper function # do something before invoation
does things before result = func_to_decorate(*args, **kwargs)
and after the function
is called here. # do something after
return result
# update wrapper.__doc__ and .func_name
# or functools.wraps
return wrapper
When decorated function is
called decorator returns wrapper
Result is returned when
the wrapper is done
http://pydanny-event-notes.readthedocs.org/en/latest/SCALE10x/python-decorators.html#decorator-template
Daniel Greenfeld
pydanny.com / @pydanny
65. Decorator
implementation
Datastore
def memoize(func):
cache = {} Return value if
args in cache
def memoized(*args):
if args in cache:
return cache[args]
result = cache[args] = func(*args) set cache
return result
Return value
return memoized
Return function
@memoize
def allcaps(string):
return string.upper()
Daniel Greenfeld
pydanny.com / @pydanny
70. multiplier decorator Multiplier function
@multiplier(5) sets the state for
def allcaps(string): the multiple
return string.upper() argument
def multiplier(multiple): Wrapper function does:
def decorator(function):
def wrapper(*args, **kwargs):
return function(*args, **kwargs) * multiple
return wrapper
return decorator Result is returned when the
wrapper is done.
When decorated function is
called the decorator function
returns the wrapper function
What am I supposed
to highlight?
Daniel Greenfeld
pydanny.com / @pydanny
79. More controversy
Decorators are
easy to explain!
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Although practicality beats purity. Decorators are
hard to explain!
Daniel Greenfeld
pydanny.com / @pydanny
80. The last section
Part IV
Daniel Greenfeld
pydanny.com / @pydanny
81. Getting it done
vs.
Technical debt
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
Daniel Greenfeld
pydanny.com / @pydanny
82. Getting it done
vs.
Technical debt
Now is better than never.
Although never is often better than *right* now.
Namespaces are one honking great idea -- let's do more of those!
Daniel Greenfeld
pydanny.com / @pydanny
83. Getting it done
vs.
Technical debt
Now is better than never.
Although never is often better than *right* now.
Namespaces are one honking great idea -- let's do more of those!
Daniel Greenfeld
pydanny.com / @pydanny
84. Some things take time
• Tests
• Documentation
Daniel Greenfeld
pydanny.com / @pydanny
85. Some things take time
Risk: Multiple coding standards
• Tests Risk: Deploying broken code
• Documentation Risk: problems upgrading dependencies
(You can skip them)
Risk: Forgetting install/deploy
Daniel Greenfeld
pydanny.com / @pydanny
86. Must-have
Documentation
• Installation/Deployment procedures
• Coding standards
• How to run tests
• Version (including __version__)
Daniel Greenfeld
pydanny.com / @pydanny
87. Easy Test Patterns
For developers racing to meet deadlines:
• Always make sure your test
harness can run
• Try using tests instead of the shell/repl.
• After the first deadline, reject any incoming
code that drops coverage.
• Use coverage.py
Daniel Greenfeld
pydanny.com / @pydanny
89. Namespaces
import re
import os
from django import forms
from myproject import utils
from twisted.internet import protocol, reactor
• Extremely powerful
• Useful
• Precise
Daniel Greenfeld
pydanny.com / @pydanny
90. import * makes
development faster[1]
from re import *
from os import *
from twisted import *
from django.forms import *
from myproject.utils import *
• Extremely powerful
• Useful
• Imports everything at once! [2]
[1]Warning: import * can be dangerous
[2]Warning: import * can be dangerous Daniel Greenfeld
pydanny.com / @pydanny
91. Comparing two modules
def compare(mod1, mod2):
title = 'nComparing {0}, {1}:'.format(
mod1.__name__,
mod2.__name__
)
print(title)
for x in dir(mod1):
for y in dir(mod2):
if x == y and not x.startswith('_'):
print("* " + x)
Daniel Greenfeld
pydanny.com / @pydanny
92. Comparing two modules
from re import *
from os import *
>>> import re
>>> import os >>> re.sys == os.sys
True
>>> compare(os, re)
Comparing os, re: >>> re.error == os.error
* sys False
* error
import * can get you into trouble
Daniel Greenfeld
pydanny.com / @pydanny
93. Breaking built-ins
def compare_builtins(mod1):
print("nComparing {0} to builtins:".format(mod1.__name__))
for x in dir(mod1):
for y in dir(globals()['__builtins__']):
if x == y and not x.startswith('_'):
print("* GLOBAL: {0}".format(x))
Checks to see if a module has items
that match any Python built-in.
Daniel Greenfeld
pydanny.com / @pydanny
94. Breaking built-ins
from re import *
from os import *
>>> compare_builtins(re) >>> compare_builtins(os)
Comparing re to builtins: Comparing os to builtins:
* GLOBAL: compile * GLOBAL: open
Breaks compile() built-in. Breaks open() built-in.
Annoying but
This can drive you crazy.
infrequent problem.
Daniel Greenfeld
pydanny.com / @pydanny
95. The open() story
before
Help on built-in function open in module __builtin__:
open(...)
open(name[, mode[, buffering]]) -> file object
Open a file using the file() type, returns a file object. This is the
preferred way to open a file. See file.__doc__ for further information.
after from os import *
Help on built-in function open in module posix:
Breaks
open(...) all
open(filename, flag [, mode=0777]) -> fd the
things!
Open a file (for low level IO).
Daniel Greenfeld
pydanny.com / @pydanny
96. Contention
import * is not for beginners.
import * is people who really know Python.
__all__ = ["echo", "surround", "reverse"]
Daniel Greenfeld
pydanny.com / @pydanny
97. Summary
Daniel Greenfeld
pydanny.com / @pydanny
98. The Zen of Python
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
Daniel Greenfeld
pydanny.com / @pydanny
99. Thank you
• Richard Jones • PyCon Poland
• Raymond Hettiger • Filip Kłębczyk
• Matt Harrison • Piotr Kasprzyk
• Kenneth Love • Lennart Regebro
• Audrey Roy
Daniel Greenfeld
pydanny.com / @pydanny