1. Hear no evil, see no evil,
patch no evil: Or, how to
monkey-patch safely.
Graham Dumpleton
@GrahamDumpleton
PyCon Australia - August 2016

2. Decorators are useful!

3. Decorators are easy to
implement?

4. Are you sure?

5. Typical decorator.
def function_wrapper(wrapped):
def _wrapper(*args, **kwargs):
return wrapped(*args, **kwargs)
return _wrapper
@function_wrapper
def function():
pass

6. This breaks
introspection.

7. __name__ and __doc__
attributes are not
preserved.

8. Doesn’t @functools.wraps() help?
import functools
def function_wrapper(wrapped):
@functools.wraps(wrapped)
def _wrapper(*args, **kwargs):
return wrapped(*args, **kwargs)
return _wrapper
@function_wrapper
def function():
pass

9. No, it doesn’t solve all
problems.

10. Still issues with: introspection,
wrapping decorators
implemented using descriptors,
and more.

11. http://blog.dscpl.com.au
Quick Link: Decorators and monkey patching.
Complicated details removed ….
Get all the details at:

12. Please try not to
implement decorators
yourself.

13. What is the solution?

14. Use ‘wrapt’.

15. Basic decorator.
import wrapt
@wrapt.decorator
def pass_through(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)
@pass_through
def function():
pass

16. Universal decorator.
import wrapt
import inspect
@wrapt.decorator
def universal(wrapped, instance, args, kwargs):
if instance is None:
if inspect.isclass(wrapped):
# Decorator was applied to a class.
return wrapped(*args, **kwargs)
else:
# Decorator was applied to a function or staticmethod.
return wrapped(*args, **kwargs)
else:
if inspect.isclass(instance):
# Decorator was applied to a classmethod.
return wrapped(*args, **kwargs)
else:
# Decorator was applied to an instancemethod.
return wrapped(*args, **kwargs)

17. Bonus feature of wrapt
if using multithreading.

18. Synchronise function calls.
from wrapt import synchronized
@synchronized # lock bound to function1
def function1():
pass
@synchronized # lock bound to function2
def function2():
pass

19. Methods of classes as well.
from wrapt import synchronized
class Class(object):
@synchronized # lock bound to instance of Class
def function_im(self):
pass
@synchronized # lock bound to Class
@classmethod
def function_cm(cls):
pass
@synchronized # lock bound to function_sm
@staticmethod
def function_sm():
pass

20. Synchronise block of code.
from wrapt import synchronized
class Object(object):
@synchronized
def function_im_1(self):
pass
def function_im_2(self):
with synchronized(self):
pass
def function_im_3(self):
with synchronized(Object):
pass

21. Don’t trust me when I say
you should use wrapt?

22. Potential candidate for being
included in the Python
standard library.
So it must be awesome.

23. Primary purpose of the
wrapt package wasn’t as
way to build decorators.

24. Primary reason for
existence of wrapt was to
help with monkey patching.

25. Decorators rely on similar
principles to monkey
patching.

26. Before decorators.
# python 2.4+
@function_wrapper
def function():
pass
# python 2.3
def function():
pass
function = function_wrapper(function)

27. Decorators are applied
when code is defined.

28. Monkey patching is
performed after the fact,
… and can’t use the
decorator syntax

29. Why monkey patch?
• Fix bugs in code you can’t modify.
• Replace/mock out code for testing.
• Add instrumentation for monitoring.

30. Monkey patching with wrapt.
# example.py
class Example(object):
def name(self):
return 'name'
# patches.py
import wrapt
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)
from example import Example
wrapt.wrap_function_wrapper(Example, 'name', wrapper)

31. Don’t patch it yourself.
# patches.py
import wrapt
@wrapt.decorator
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)
from example import Example
# DON’T DO THIS.
Example.name = wrapper(Example.name)

32. Direct patching of
methods breaks in
certain corner cases.
Let wrapt apply the
wrapper for you.

33. Avoiding imports.
# patches.py
import wrapt
@wrapt.patch_function_wrapper('example', 'Example.name')
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)

34. What about testing,
where we do not want
permanent patches?

35. Mock alternative.
# example.py
class Storage(object):
def lookup(self, key):
return 'value'
def clear(self):
pass
# tests.py
import wrapt
@wrapt.transient_function_wrapper('example', 'Storage.lookup')
def validate_storage_lookup(wrapped, instance, args, kwargs):
assert len(args) == 1 and not kwargs
return wrapped(*args, **kwargs)
@validate_storage_lookup
def test_method():
storage = Storage()
result = storage.lookup('key')
storage.clear()

36. What if we need to
intercept access to single
instance of an object?

37. Transparent object proxy.
# tests.py
import wrapt
class StorageProxy(wrapt.ObjectProxy):
def lookup(self, key):
assert isinstance(key, str)
return self.__wrapped__.lookup(key)
def test_method():
storage = StorageProxy(Storage())
result = storage.lookup(‘key')
storage.clear()

38. Beware though of
ordering problems when
applying monkey patches.

39. Import from module.
# example.py
def function():
pass
# module.py
from example import function
# patches.py
import wrapt
@wrapt.patch_function_wrapper('example', 'function')
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)

40. Plus importing and patching
of modules that the
application doesn’t need.

41. Post import hooks (PEP 369).
# patches.py
import wrapt
def wrapper(wrapped, instance, args, kwargs):
return wrapped(*args, **kwargs)
@wrapt.when_imported('example')
def apply_patches(module):
wrapt.wrap_function_wrapper(module, 'Example.name', wrapper)

42. Better, but still requires
patches module to be
imported before anything
else in the main
application script.

43. Need a way to trigger
monkey patches without
modifying application code.

44. The autowrapt package.
pip install autowrapt

45. Bundle patches as module.
from setuptools import setup
PATCHES = [
'wsgiref.simple_server = wrapt_wsgiref_debugging:apply_patches'
]
setup(
name = 'wrapt_wsgiref_debugging',
version = '0.1',
py_modules = ['wrapt_wsgiref_debugging'],
entry_points = {‘wrapt_wsgiref_debugging': PATCHES}
)

46. Patch to time functions.
from __future__ import print_function
from wrapt import wrap_function_wrapper
from timeit import default_timer
def timed_function(wrapped, instance, args, kwargs):
start = default_timer()
print('start', wrapped.__name__)
try:
return wrapped(*args, **kwargs)
finally:
duration = default_timer() - start
print('finish %s %.3fms' % (
wrapped.__name__, duration*1000.0))
def apply_patches(module):
print('patching', module.__name__)
wrap_function_wrapper(module,
'WSGIRequestHandler.handle', timed_function)

47. Enabling patches.
$ AUTOWRAPT_BOOTSTRAP=wrapt_wsgiref_debugging
$ export AUTOWRAPT_BOOTSTRAP
$ python app.py
patching wsgiref.simple_server
start handle
127.0.0.1 - - [14/Jul/2016 10:18:46] "GET / HTTP/1.1" 200 12
finish handle 1.018ms

48. Packaging of patches means
they could technically be
shared via PyPi.
Eg: instrumentation for
monitoring.

49. Reasons to use wrapt.
• Create better decorators.
• Awesome thread synchronisation decorator.
• Safer mechanisms for monkey patching.

50. Graham.Dumpleton@gmail.com
@GrahamDumpleton
wrapt.readthedocs.io
blog.dscpl.com.au