Computable Content

1. Computable Content:   Lessons Learned Paco Nathan @pacoid Director, Learning Group @ O’Reilly Media 2017-‐06-‐21

2. An observation 2

3. Oriole 3

4. One approach 4

5. Jupyter use @ O’Reilly Media ▪ Embracing Jupyter notebooks at O’Reilly  oreilly.com/ideas/jupyter-‐at-‐oreilly ▪ Learning alongside innovators, thought-‐by-‐thought, in context  oreilly.com/ideas/oreilly-‐oriole-‐learn-‐alongside-‐innovators-‐ thought-‐by-‐thought-‐in-‐context ▪ Oriole online tutorials  safaribooksonline.com/oriole/ ▪ How do you learn?  oreilly.com/learning/how-‐do-‐you-‐learn 5

6. For example ▪ A unique new medium blends code, data, text,   and video into a narrated learning experience  with computable content ▪ Purely browser-‐based UX; zero installation   required ▪ Substantially higher engagement metrics ▪ Opens the door for live coding in assessments 6 oreilly.com/learning/regex-‐golf-‐with-‐peter-‐norvig

7. Motivations O’Reilly needed a way for authors to use Jupyter notebooks to create professional publications. We also wanted to integrate video narration into the UX. The result is a unique new medium called Oriole: ▪ Context as a “unit of thought” ▪ Code and video sync’ed together ▪ Each web session get its own Docker container in the cloud ▪ 100% HTML experience, no download/install/config needed ▪ Jupyter notebooks used in the middleware ▪ Leverage interactive, data-‐driven graphics 7

8. Outcomes 8 ▪ Tutorials are now much quicker to publish than “traditional” books and videos ▪ Less time required for innovators in programming, data science, devops, design, etc. – who tend to be really busy people ▪ Audience gets direct, hands-‐on, contextualized experience across a wide variety of programming environments

9. Limitations 9 ▪ Notebook kernels run REPLs, so older languages were not feasible ▪ Brief code blocks with tangible outcomes –   precludes business topics, systems engineering, etc ▪ What materials will fit within a Docker container?

10. Third iteration of Jupyter @ O’Reilly 10 1. notebooks as supplemental material to other published work 2. notebooks published as HTML, as articles 3. computable content, containerized notebooks + video narratives 4. hosted notebooks

11. Long-‐term goal: make learning materials more powerful by integrating compute engines + data services 11

12. Project Jupyter 12

13. Project Jupyter ▪ The evolution of iPython notebooks, applied to a range of different programming languages and environments ▪ https://jupyter.org/ ▪ https://github.com/ipython/ipython/wiki/ IPython-‐kernels-‐for-‐other-‐languages 13

14. Example Notebook 14

15. Projects 15 ▪ JupyterHub  github.com/jupyterhub/jupyterhub ▪ Jupyter in Education  groups.google.com/forum/#!forum/ jupyter-‐education ▪ JupyterLab  github.com/jupyterlab/jupyterlab ▪ Jupyter Kernels  github.com/ipython/ipython/wiki/ IPython-‐kernels-‐for-‐other-‐languages

16. A suite of network protocols Think of Jupyter, at its core, as a suite of network protocols: Jupyter is to the remote semantics of a REPL as… HTTP is to the remote semantics of file share 16

17. A suite of network protocols 17 Code%runs% in%a%REPL Kernel Edi0ng%+% Results Notebook Network Protocol

18. History, Context 18

19. Notebook metaphor Wolfram Research introduced notebooks in 1988 for working   with Mathematica 19

20. Related work 20

21. Literate programming Don Knuth  literateprogramming.com/ Paraphrased:  Instead of telling computers what to do, tell other people what you want the computers to do 21

22. Speech acts PyCon 2016 Keynote, Lorena Barba youtu.be/ckW1xuGVpug?t=35m11s  (video) figshare.com/articles/ PyCon2016_Keynote/3407779  (slides) Highly recommended: speech acts   (based on Winograd and Flores)   as theory here 22

23. Best Practices 23

24. The following lessons learned in using Jupyter notebooks + video for learning materials apply well in many situations for data science teams working across an organization 24

25. Teaching with Jupyter – 1 of 2 ▪ focus on a concise “unit of thought” ▪ invest the time and editorial effort to create a good intro ▪ keep your narrative simple and reasonably linear ▪ “chunk” the text and code into understandable parts ▪ alternate between text, code, output, further links, etc. ▪ code cells should be brief (< 10 lines), must show output 25

26. Teaching with Jupyter – 2 of 2 ▪ load data+libraries from the container, not the network ▪ clear all output then “Run All” – or it didn’t happen ▪ video narratives: there’s text, and there’s subtext... ▪ pause after each “beat”: smile, breathe, let people follow you For JVM people: stop thinking only about IDEs, Ivy, Maven, etc. (ibid, Knuth1984)  BUILD UBER JARS, LOAD LIBS FROM CONTAINER, NOT THE NETWORK!  (apologies for shousng) 26

27. Sharing is caring In data science, we see the benefits to teams for shared insights, storytelling, etc. Meanwhile domain expertise is generally more important than knowledge about tools There’s a value for developers to use notebooks in lieu of IDEs in some cases – what are those cases? GitHub now renders notebooks, so they can be used for documentation, reporting, etc. Digital Object Identifiers (DOI) can be assigned through Zenodo, making notebooks citable for academic publication 27

28. Authoring and Scale-‐Out 28

29. Launchbot.io 29

30. Achieving scale ▪ Launchbot.io allows a notebook author to build a container that includes the required Jupyter kernel, installed libraries, datasets, etc. ▪ Install Docker on your laptop ▪ Backend uses Git and DockerHub to manage containers ▪ For scale, deploy to DC/OS or a cloud 30

31. “A notebook, a container, and ~20 minutes   of informal video walk into a bar…” 31

32. System architecture 32 Tutorial Middleware Cluster

33. O’Reilly Strata NY, Sep 25-‐28  SG, Dec 4-‐7 O’Reilly ArXﬁcial Intelligence NY, Jun 26-‐29  SF, Sep 17-‐20 JupyterCon NY, Aug 22-‐25 33

34. 34 Learn Alongside  Innovators Just Enough Math Building Data Science Teams Hylbert-‐Speys How Do You Learn? periodic newslewer with updates,   events, conference summaries… liber118.com/pxn/  @pacoid