July Tech Festa 2016 #jtf2016 で発表させていただいたスライドです。 NIIクラウド運用で試行してみている、文芸的機械化の試みについてご紹介。

1. Jupyter Notebook Ansible

2. ◦
◦ GitHub/Twitter: @yacchin1205
:
:
: ( )
(NII)
( )

3. ◦ /

4.
is

6. …
1867 1930
…
– – , 1978

7. …
…
– – , 1978

8. /
/

10. Which type are you?
ANACONDA EMPOWERS THE ENTIRE DATA SCIENCE TEAM
<https: //www.continuum.io/open-data-science>
as of July 1, 2016

11. ( )
Moving Target
◦
◦
◦
◦
◦
→

12. NII
• …
•
• Hadoop …
• …
•
•
•
•

13. Hardiman, General Electric in 1965
https://www.engadget.com/2014/01/26/ge-man-amplifying-robots/

14. Literate
Computing
for
Reproducible
Infrastructure

15. Jupyter: Literate Computing

16. Jupyter
Data Scientist
Jupyter Notebook
https://tmpnb.org

17. Jupyter( IPython) + Ansible
◆
◆
◆
Notebook
Literate Computing for Reproducible Infrastructure

21. “Literate Computing for Reproducible Infrastructure”
/
= computation = communication
◦ data live-code
◦

22. Automation .. Communication
:
: Communication
◦
◦
◦
(Code )
◦ /

23. … Traceability
… Reusability
… Reproducibility

24. “ ”Notebook
https://github.com/NII-cloud-operation/Literate-computing-Basics

26. Hadoop

28. Notebook

29. Ansible Playbook

30. Ansible Playbook

31. …

32. …

34. Update

37. •
• 1
• Notebook
• Notebook
• Notebook
…

39. • https://github.com/ansible/ansible/issues/13485
Not only the handler is not triggered,
it does not throw any error if it does not exist.

40. • …

41. Notebook

42. Jupyter + Ansible

43. Jupyter Notebook
http://qiita.com/yacchin1205/items/efaa498ca68e7c169015

44. Ansible
• Ansible
•
• / sudo
•
• Notebook
•
Notebook
•

45. Jupyter

46. N.
)
takasix
2010 SIer SaaS
2011 ( )
2012 ( ) Microsoft Kinect Gescha
2013 2015 SIer OSS”CloudConductor-Patterns”
2016

47. Jupyter Notebooks Architecture
0MQ
HTTP &
Websokets

48. notebook extension system
0MQ
HTTP &
Websokets
nbextensions serverextensions

49. nbextensions
• JavaScript
• UI, Note
•
•

50. server extension
• 4.2
• Notebook server
• HTTP
• (?)

51. Jupyter Notebook

52. nb extension
• Jupyter_code_cell_status
• https://github.com/NII-cloud-operation/Jupyter-code_cell_status
• Jupyter-multi_outputs
• https://github.com/NII-cloud-operation/Jupyter-multi_outputs

53. Jupyter-code_cell_status

54. Jupyter_code_cell_status -
•

55. Jupyter_code_cell_status -

56. Jupyter_code_cell_status -
•

57. Jupyter_code_cell_status -
• nb extension

58. Jupyter-multi_outputs

59. Jupyter-multi_outputs -
•
•
• ”Trial and Error”
•
• :
• :

60. Jupyter-multi_outputs -

61. Jupyter-multi_outputs - (1)
•

62. Jupyter-multi_outputs - (2)
•

63. Jupyter-multi_outputs - (3)
•

64. Jupyter-multi_outputs - (4)
• .ipynb
• Extension
( )

65. Jupyter-multi_outputs -
• diff
• ……

66. Jupyter-multi_outputs -
• 1 stream

67. Jupyter-multi_outputs -
%matplotlib inline
import numpy
from matplotlib import pyplot
x = numpy.arange(0, 10, 0.2)
y = numpy.cos(x)
pyplot.plot(x,y)
display_data:[<matplotlib.lines.Line2D at 0x7eff779f56d0>]
execute_result:data:image/png;base64,iVBORw0KGgoA…

68. notebook extension system

69. notebook extension system (1)
• nb extensions
• Extension
• JavaScript prototype Extension
✴ code_cell_status multi_outputs
1 Extension

70. notebook extension system (2)
• server extensions
•
• notebook extension

71.
Jupyter notebook
LiterateComputing
for Reproducible Infrastructure
PullRequest

72. Jupyter Notebook Ansible