Slides for my talk at Women Who Code Talk night in Berlin 17/09/2019
https://www.meetup.com/de-DE/Women-Who-Code-Berlin-Germany/events/264126194/
Jupyter notebooks are a great tool for interactive programming but they
can also quickly turn messy. With their focus on rapid prototyping, it
can take some discipline to not end up with a pile of notebooks full of
messy code. On top of that, jupyter notebooks are difficult to version
control and it thus becomes hard to track changes. It is threfore easy
to end up in a situation where you don’t understand your own analysis
from some months ago, not to mention the code from your colleague.
In this talk, I will share a few simple and practical tips that I found
helpful to improve the notebook workflow in our data science team.
Link to Jupyter Savehook Gist: https://gist.github.com/corriebar/cca8a8536bfa04cfa5ea745587b99146
3. Messy?
3
● Easy to end up with a messy
notebook pile
● Rapid prototyping leads to less
documentation
Where is the
PyTorch import?
● Reproducibility of results
Which data was
used here?
4. Collaboration?
4
● Will future-you still understand
your notebooks?
● Or your colleague?
Where is the
PyTorch import?
● Will they be able to run the
notebooks?
Which data was
used here?
7. Version Control!
7
jupyter nbconvert --to md notebook.ipynb
● Convert notebooks to
markdown
● Using nbconvert*
● Can be automated via
Jupyter SaveHook
*Other options are for example: jupytext, reviewnb
9. Collaboration!
9
● Use a code (and analysis)
review process
● In our team, we use the GitHub
workflow
● A review encourages clean-ups
and documentation
11. Some more tidying
11
● Include a TL;DR summary of the question
you’re trying to solve and your conclusion
Cookiecutter Data Science:
● Introduce a naming convention for
notebooks, e.g. 1.0-cba-initial-data-exploration
● Use a default folder structure
12. Summary
12
● Collaborate and use Code Reviews
● Version Control with converted notebooks
● Make notebooks reproducible
● Clean up your notebooks
