Slides from my "It's 2019 & I'm still using Python 2! Should I be worried?" presentation, where I'm talking about different strategies how to prepare to the Python 2 EOL.
8. @SebaWitowski bit.ly/it-is-2019
It’s not “How we migrated to
Python 3 at FooBar company”
• Migrating Pinterest from Python2 to Python3 - PyCon 2019
• Instagram Makes a Smooth Move to Python 3
• How Dropbox migrated to Python 3
• Python 2 is dead! Drag your old code into the modern age
10. @SebaWitowski bit.ly/it-is-2019
We will talk about
• Risks of staying on an unsupported Python version
• Different possible solutions for your project
• And finally - the migration itself
14. @SebaWitowski bit.ly/it-is-2019
• The cost of migration is too high from a
business point of view
• There is always a new feature or an urgent fix
Why Python 2 is still used?
18. @SebaWitowski bit.ly/it-is-2019
• The cost of migration is too high from a
business point of view
• There is always a new feature or an urgent fix
• Migrating an old codebase can be scary
Why Python 2 is still used?
21. @SebaWitowski bit.ly/it-is-2019
Benefits of migrating to Python 3
• You won’t be using an unsupported version of
a programming language in a few months
• Python 3 is faster than Python 2 …
23. @SebaWitowski bit.ly/it-is-2019
Benefits of migrating to Python 3
• You won’t be using an unsupported version of
a programming language in a few months
• Python 3 is faster than Python 2 …
• … and it has new features (asyncio, type hints,
ordered dictionaries, f-strings, better unicode)!
24. @SebaWitowski bit.ly/it-is-2019
Benefits of migrating to Python 3
• You won’t be using an unsupported version of
a programming language in a few months
• Python 3 is faster than Python 2 …
• … and it has new features (asyncio, type hints,
ordered dictionaries, f-strings, better unicode)!
• Your dependencies are already on Python 3 …
25. @SebaWitowski bit.ly/it-is-2019
Benefits of migrating to Python 3
• You won’t be using an unsupported version of
a programming language in a few months
• Python 3 is faster than Python 2 …
• … and it has new features (asyncio, type hints,
ordered dictionaries, f-strings, better unicode)!
• Your dependencies are already on Python 3 …
• … and they will stop supporting Python 2 soon!
27. @SebaWitowski bit.ly/it-is-2019
Benefits of migrating to Python 3
• You won’t be using an unsupported version of
a programming language in a few months
• Python 3 is faster than Python 2 …
• … and it has new features (asyncio, type hints,
ordered dictionaries, f-strings, better unicode)!
• Your dependencies are already on Python 3 …
• … and they will stop supporting Python 2 soon!
Show this slide to your manager!
29. @SebaWitowski bit.ly/it-is-2019
• You dependencies are probably already migrated
(which was not the case a few years ago)
Now is the best time to migrate!
31. @SebaWitowski bit.ly/it-is-2019
• Your dependencies are probably already migrated
(which was not the case a few years ago)
Now is the best time to migrate!
32. @SebaWitowski bit.ly/it-is-2019
• Your dependencies are probably already migrated
(which was not the case a few years ago)
• The automatic tools have been tested by
thousands of developers
Now is the best time to migrate!
33. @SebaWitowski bit.ly/it-is-2019
• You dependencies are probably already migrated
(which was not the case a few years ago)
• The automatic tools have been tested by
thousands of developers
• You can learn from others …
Now is the best time to migrate!
34. @SebaWitowski bit.ly/it-is-2019
• You can learn from others:
• https://blogs.dropbox.com/tech/2019/02/
incrementally-migrating-over-one-million-lines-
of-code-from-python-2-to-python-3/
• https://docs.python.org/3/howto/pyporting.html
• https://www.youtube.com/watch?
v=66XoCk79kjM
Now is the best time to migrate!
36. @SebaWitowski bit.ly/it-is-2019
• You dependencies are probably already migrated
(which was not the case a few years ago)
• The automatic tools has been tested by thousands
of developers
• You can learn from others
• And you still have those 2-3 months of “runway”
Now is the best time to migrate!
41. @SebaWitowski bit.ly/it-is-2019
Commercial vendors
You won’t get support from the Python community,
but you might get it from commercial vendors!
E.g. RHEL7 that was released in 2014 is still using
Python 2.7, so Red Hat should provide you support
until 2024 (based on their 10-year support lifecycle).
If you are willing to pay for it.
47. @SebaWitowski bit.ly/it-is-2019
• Do nothing
• Freeze the state of your application (docker)
• Change Python interpreter
• Maintain your own Python 2 build
• Migrate to Python 3
• Rewrite you application
Low
High
Effort
What can you do about Python 2 EOL?
48. @SebaWitowski bit.ly/it-is-2019
• Do nothing
• Freeze the state of your application (docker)
• Change Python interpreter
• Maintain your own Python 2 build
• Migrate to Python 3
• Rewrite you application
Low
High
Effort
What can you do about Python 2 EOL?
54. @SebaWitowski bit.ly/it-is-2019
Freeze the state of your application
• Write a Dockerfile that uses Python 2 as the base image
• Add all dependencies
• Set your app as a Docker image
• Push that image to a public/private docker repository
• Now you have an immutable container with your app!
56. @SebaWitowski bit.ly/it-is-2019
• Do nothing
• Freeze the state of your application (docker)
• Change Python interpreter
• Maintain your own Python 2 build
• Migrate to Python 3
• Rewrite you application
Low
High
Effort
What can you do about Python 2 EOL?
62. @SebaWitowski bit.ly/it-is-2019
• Do nothing
• Freeze the state of your application (docker)
• Change Python interpreter
• Maintain your own CPython 2 build
• Migrate to Python 3
• Rewrite you application
Low
High
Effort
What can you do about Python 2 EOL?
63. @SebaWitowski bit.ly/it-is-2019
• Do nothing
• Freeze the state of your application (docker)
• Change Python interpreter
• Maintain your own CPython 2 build
• Migrate to Python 3
• Rewrite you application
Low
High
Effort
What can you do about Python 2 EOL?
65. @SebaWitowski bit.ly/it-is-2019
Migrate to Python 3
• With straddling code (code that supports
both Python 2 and Python 3)
• Run tests under Python 3 (they will fail)
• Rewrite your application until it works
under Python 2 and 3
• Switch Python version in production
• [OPTIONAL] Remove the Python 2 code
66. @SebaWitowski bit.ly/it-is-2019
Migrate to Python 3
• With straddling code (code that supports
both Python 2 and Python 3)
• Rewriting Python 2 code to Python 3
• Less work (you don’t care about Python 2)
• Scary to switch (what if you didn’t test
something?)
• Requires feature freeze
67. @SebaWitowski bit.ly/it-is-2019
• Do nothing
• Freeze the state of your application (docker)
• Change Python interpreter
• Maintain your own CPython 2 build
• Migrate to Python 3
• Rewrite your application
Low
High
Effort
What can you do about Python 2 EOL?
77. @SebaWitowski bit.ly/it-is-2019
“Hey boss, before we spend those 3 months
migrating our project to Python 3, we need to
spend 2 more weeks writing tests in Python 2.
We will have to migrate those tests as well.”
79. @SebaWitowski bit.ly/it-is-2019
1. You need tests
1. Write your tests so they work with Python 2 and 3
2. If you absolutely can’t afford to write tests and your
project is a web application - record functional
tests
81. @SebaWitowski bit.ly/it-is-2019
Record your tests in Selenium
• Record every possible use case in your application:
• Click every button
• Fill in every form
• Make mistakes to see error messages
• Use assertions to check if data was preserved
correctly (string encoding will be one of the main
problems during the migration)!
88. @SebaWitowski bit.ly/it-is-2019
2. You need tests
If you have time to actually write tests focus on:
1. Unicode support (test areas where the strings are
getting IN and OUT of your application)
2. Division (test functions that are dividing numbers -
especially when dealing with money!)
90. @SebaWitowski bit.ly/it-is-2019
Set up tox
• Install: pip install tox
• Create tox.ini file:
• Make sure you have Python 2.7 and 3.7 installed!
• Run tox command
# content of: tox.ini , put in same dir as setup.py
[tox]
envlist = py27,py37
skipsdist = True
[testenv]
# install pytest in the virtualenv where commands will be executed
deps = pytest
commands =
# NOTE: you can run any command line tool here - not just tests
python -m pytest
100. @SebaWitowski bit.ly/it-is-2019
Automatic tools
They can do most of the repetitive work
(replace print statement with function, change
modules to their Python 3 equivalents, etc.).
But their output has to be checked by you!
106. @SebaWitowski bit.ly/it-is-2019
Futurize in practice
Fixes are divided into 2 categories:
• Stage 1 - “safe” modifications (don’t break Python 2.6
compatibility, don’t introduce new dependencies)
• Stage 2 - others (introduce dependencies from future
package, rename standard library imports to Python 3
versions, etc.)
107. @SebaWitowski bit.ly/it-is-2019
Futurize in practice
You can run the whole stages of fixes at once:
• futurize --stage1
• futurize --stage2
• But it’s better to run fixes one by one
• Don’t mix automatic fixes with manual changes!
109. @SebaWitowski bit.ly/it-is-2019
Futurize in practice
Next, you have to decide what to do with strings:
A good explanation of Unicode problems and how to
solve them:
https://nedbatchelder.com/text/unipain.html
from builtins import bytes, str
b = bytes(b'x00ABCD')
s = str(u'This is normal text')
110. @SebaWitowski bit.ly/it-is-2019
A sane approach to Unicode
“You should have bytes on the outside of your system
and Unicode on the inside.
You should perform encode/decode on the edges of
your system.”
113. @SebaWitowski bit.ly/it-is-2019
Some good resources that can help you:
• http://python3porting.com/toc.html
• http://blog.pyspoken.com/2018/02/13/python-2-
to-3-migration-guide/
• https://portingguide.readthedocs.io
• YouTube videos from conferences
More resources:
115. @SebaWitowski bit.ly/it-is-2019
Conclusions
• But now is the best time to migrate:
• Your dependencies are already migrated
• Migration tools are mature and well tested
• Python 2 is still supported for a while