O slideshow foi denunciado.

Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.

1

Compartilhar

Carregando em…3
×
1 de 50
1 de 50

Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.

1

Compartilhar

Baixar para ler offline

Python is a dynamic programming language and has a strong tradition of adhering to a programming style called duck-typing. This means that it is possible to easily modify an application's code while it is running. One might wish to do this for various reasons, including enhancing the functionality of code, correcting errant behaviour, or adding instrumentation or debugging code.

Making such code modifications can be tricky though and not done correctly can potentially interfere with the operation of the original code, through destroying introspection abilities, not honouring the duck-typing mantra or due to being applied at the wrong time.

If you do need to do monkey patching though, the 'wrapt' library is your friend, with its transparent object proxy wrappers and post import hook mechanism, it allows you to safely monkey patch code to modify its behaviour.

Learn about the 'wrapt' library and the joys, but also the dangers, of monkey patching.

Python is a dynamic programming language and has a strong tradition of adhering to a programming style called duck-typing. This means that it is possible to easily modify an application's code while it is running. One might wish to do this for various reasons, including enhancing the functionality of code, correcting errant behaviour, or adding instrumentation or debugging code.

Making such code modifications can be tricky though and not done correctly can potentially interfere with the operation of the original code, through destroying introspection abilities, not honouring the duck-typing mantra or due to being applied at the wrong time.

If you do need to do monkey patching though, the 'wrapt' library is your friend, with its transparent object proxy wrappers and post import hook mechanism, it allows you to safely monkey patch code to modify its behaviour.

Learn about the 'wrapt' library and the joys, but also the dangers, of monkey patching.

Mais Conteúdo rRelacionado

Livros relacionados

Gratuito durante 14 dias do Scribd

Ver tudo

Audiolivros relacionados

Gratuito durante 14 dias do Scribd

Ver tudo

Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely.

  1. 1. Hear no evil, see no evil, patch no evil: Or, how to monkey-patch safely. Graham Dumpleton @GrahamDumpleton PyCon Australia - August 2016
  2. 2. Decorators are useful!
  3. 3. Decorators are easy to implement?
  4. 4. Are you sure?
  5. 5. Typical decorator. def function_wrapper(wrapped): def _wrapper(*args, **kwargs): return wrapped(*args, **kwargs) return _wrapper @function_wrapper def function(): pass
  6. 6. This breaks introspection.
  7. 7. __name__ and __doc__ attributes are not preserved.
  8. 8. Doesn’t @functools.wraps() help? import functools def function_wrapper(wrapped): @functools.wraps(wrapped) def _wrapper(*args, **kwargs): return wrapped(*args, **kwargs) return _wrapper @function_wrapper def function(): pass
  9. 9. No, it doesn’t solve all problems.
  10. 10. Still issues with: introspection, wrapping decorators implemented using descriptors, and more.
  11. 11. http://blog.dscpl.com.au Quick Link: Decorators and monkey patching. Complicated details removed …. Get all the details at:
  12. 12. Please try not to implement decorators yourself.
  13. 13. What is the solution?
  14. 14. Use ‘wrapt’.
  15. 15. Basic decorator. import wrapt @wrapt.decorator def pass_through(wrapped, instance, args, kwargs): return wrapped(*args, **kwargs) @pass_through def function(): pass
  16. 16. Universal decorator. import wrapt import inspect @wrapt.decorator def universal(wrapped, instance, args, kwargs): if instance is None: if inspect.isclass(wrapped): # Decorator was applied to a class. return wrapped(*args, **kwargs) else: # Decorator was applied to a function or staticmethod. return wrapped(*args, **kwargs) else: if inspect.isclass(instance): # Decorator was applied to a classmethod. return wrapped(*args, **kwargs) else: # Decorator was applied to an instancemethod. return wrapped(*args, **kwargs)
  17. 17. Bonus feature of wrapt if using multithreading.
  18. 18. Synchronise function calls. from wrapt import synchronized @synchronized # lock bound to function1 def function1(): pass @synchronized # lock bound to function2 def function2(): pass
  19. 19. Methods of classes as well. from wrapt import synchronized class Class(object): @synchronized # lock bound to instance of Class def function_im(self): pass @synchronized # lock bound to Class @classmethod def function_cm(cls): pass @synchronized # lock bound to function_sm @staticmethod def function_sm(): pass
  20. 20. Synchronise block of code. from wrapt import synchronized class Object(object): @synchronized def function_im_1(self): pass def function_im_2(self): with synchronized(self): pass def function_im_3(self): with synchronized(Object): pass
  21. 21. Don’t trust me when I say you should use wrapt?
  22. 22. Potential candidate for being included in the Python standard library. So it must be awesome.
  23. 23. Primary purpose of the wrapt package wasn’t as way to build decorators.
  24. 24. Primary reason for existence of wrapt was to help with monkey patching.
  25. 25. Decorators rely on similar principles to monkey patching.
  26. 26. Before decorators. # python 2.4+ @function_wrapper def function(): pass # python 2.3 def function(): pass function = function_wrapper(function)
  27. 27. Decorators are applied when code is defined.
  28. 28. Monkey patching is performed after the fact, … and can’t use the decorator syntax
  29. 29. Why monkey patch? • Fix bugs in code you can’t modify. • Replace/mock out code for testing. • Add instrumentation for monitoring.
  30. 30. Monkey patching with wrapt. # example.py class Example(object): def name(self): return 'name' # patches.py import wrapt def wrapper(wrapped, instance, args, kwargs): return wrapped(*args, **kwargs) from example import Example wrapt.wrap_function_wrapper(Example, 'name', wrapper)
  31. 31. Don’t patch it yourself. # patches.py import wrapt @wrapt.decorator def wrapper(wrapped, instance, args, kwargs): return wrapped(*args, **kwargs) from example import Example # DON’T DO THIS. Example.name = wrapper(Example.name)
  32. 32. Direct patching of methods breaks in certain corner cases. Let wrapt apply the wrapper for you.
  33. 33. Avoiding imports. # patches.py import wrapt @wrapt.patch_function_wrapper('example', 'Example.name') def wrapper(wrapped, instance, args, kwargs): return wrapped(*args, **kwargs)
  34. 34. What about testing, where we do not want permanent patches?
  35. 35. Mock alternative. # example.py class Storage(object): def lookup(self, key): return 'value' def clear(self): pass # tests.py import wrapt @wrapt.transient_function_wrapper('example', 'Storage.lookup') def validate_storage_lookup(wrapped, instance, args, kwargs): assert len(args) == 1 and not kwargs return wrapped(*args, **kwargs) @validate_storage_lookup def test_method(): storage = Storage() result = storage.lookup('key') storage.clear()
  36. 36. What if we need to intercept access to single instance of an object?
  37. 37. Transparent object proxy. # tests.py import wrapt class StorageProxy(wrapt.ObjectProxy): def lookup(self, key): assert isinstance(key, str) return self.__wrapped__.lookup(key) def test_method(): storage = StorageProxy(Storage()) result = storage.lookup(‘key') storage.clear()
  38. 38. Beware though of ordering problems when applying monkey patches.
  39. 39. Import from module. # example.py def function(): pass # module.py from example import function # patches.py import wrapt @wrapt.patch_function_wrapper('example', 'function') def wrapper(wrapped, instance, args, kwargs): return wrapped(*args, **kwargs)
  40. 40. Plus importing and patching of modules that the application doesn’t need.
  41. 41. Post import hooks (PEP 369). # patches.py import wrapt def wrapper(wrapped, instance, args, kwargs): return wrapped(*args, **kwargs) @wrapt.when_imported('example') def apply_patches(module): wrapt.wrap_function_wrapper(module, 'Example.name', wrapper)
  42. 42. Better, but still requires patches module to be imported before anything else in the main application script.
  43. 43. Need a way to trigger monkey patches without modifying application code.
  44. 44. The autowrapt package. pip install autowrapt
  45. 45. Bundle patches as module. from setuptools import setup PATCHES = [ 'wsgiref.simple_server = wrapt_wsgiref_debugging:apply_patches' ] setup( name = 'wrapt_wsgiref_debugging', version = '0.1', py_modules = ['wrapt_wsgiref_debugging'], entry_points = {‘wrapt_wsgiref_debugging': PATCHES} )
  46. 46. Patch to time functions. from __future__ import print_function from wrapt import wrap_function_wrapper from timeit import default_timer def timed_function(wrapped, instance, args, kwargs): start = default_timer() print('start', wrapped.__name__) try: return wrapped(*args, **kwargs) finally: duration = default_timer() - start print('finish %s %.3fms' % ( wrapped.__name__, duration*1000.0)) def apply_patches(module): print('patching', module.__name__) wrap_function_wrapper(module, 'WSGIRequestHandler.handle', timed_function)
  47. 47. Enabling patches. $ AUTOWRAPT_BOOTSTRAP=wrapt_wsgiref_debugging $ export AUTOWRAPT_BOOTSTRAP $ python app.py patching wsgiref.simple_server start handle 127.0.0.1 - - [14/Jul/2016 10:18:46] "GET / HTTP/1.1" 200 12 finish handle 1.018ms
  48. 48. Packaging of patches means they could technically be shared via PyPi. Eg: instrumentation for monitoring.
  49. 49. Reasons to use wrapt. • Create better decorators. • Awesome thread synchronisation decorator. • Safer mechanisms for monkey patching.
  50. 50. Graham.Dumpleton@gmail.com @GrahamDumpleton wrapt.readthedocs.io blog.dscpl.com.au

×