SlideShare uma empresa Scribd logo
1 de 10
Baixar para ler offline
USING DOCKER FOR DATA 
SCIENCE
WHY DOCKER 
Portable environment 
Isolated between projects 
Stateless 
Fast local file access 
Hetrogenous
GET DOCKER 
https://docs.docker.com/installation/ 
boot2docker .dmg or .exe 
apt-get install docker.io ...
RUN SCIPYSERVER 
$ docker run -d -e "PASSWORD=YourPassword?" ipython/scipyserver 
$ docker run  
-d  
-e "PASSWORD=YourPassword?"  
--name dev_notebook  
-p 443:8888  
ipython/scipyserver 
https://localhost:443 
https://{boot2docker ip}:443
CREATE DATA-ONLY CONTAINERS 
$ docker run  
-d  
-v ~/notebooks:/notebooks  
--name notebooks_container  
ubuntu 
echo notebooks 
$ docker run -d -v ~/data:/data --name data_container ubuntu echo data
MOUNT DATA-ONLY CONTAINERS 
$ docker stop dev_notebook 
$ docker rm dev_notebook 
$ docker run  
-d  
-e "PASSWORD=YourPassword?"  
--name dev_notebook  
-p 443:8888  
--volumes-from data_container  
--volumes-from notebooks_container  
ipython/scipyserver
CREATE A DOCKERFILE 
FROM ipython/scipyserver 
MAINTAINER Calvin Giles <calvin.giles@gmail.com> 
COPY requirements.txt /requirements.txt 
RUN pip2 install -r /requirements.txt 
RUN pip3 install -r /requirements.txt 
$ docker build  
-t calvingiles/ds-notebook  
. 
$ docker run  
-d  
-e "PASSWORD=YourPassword?"  
--name dev_notebook  
-p 443:8888  
--volumes-from data_container  
--volumes-from notebooks_container  
calvingiles/ds-notebook
NEXT TIME 
Connecting to local database containers 
Tweaking the boot2docker vm memory from 2GB to 8 (or 
more...) 
Linking to private git repositories 
Automated builds with github linking
MORE? 
Docker User Guide: 
http://docs.docker.com/userguide/ 
Docker Reference: 
http://docs.docker.com/reference/commandline/cli/ 
ipython docker images: 
https://registry.hub.docker.com/repos/ipython/ 
my docker image: 
https://github.com/calvingiles/ds-notebook 
https://registry.hub.docker.com/u/calvingiles/ds-notebook/
ABOUT ME 
Calvin Giles 
Data Scientist at Adthena 
PyData Meetup Organiser 
untangleconsulting.io 
calvin.giles@gmail.com 
@calvingiles on twitter, github, docker hub (and many more)

Mais conteúdo relacionado

Mais procurados

Drupal Camp Brighton 2015: Ansible Drupal Medicine show
Drupal Camp Brighton 2015: Ansible Drupal Medicine showDrupal Camp Brighton 2015: Ansible Drupal Medicine show
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
George Boobyer
 
Ops for everyone - John Britton
Ops for everyone - John BrittonOps for everyone - John Britton
Ops for everyone - John Britton
Devopsdays
 
Docker & FieldAware
Docker & FieldAwareDocker & FieldAware
Docker & FieldAware
Jakub Jarosz
 

Mais procurados (18)

Shared Object images in Docker: What you need is what you want.
Shared Object images in Docker: What you need is what you want.Shared Object images in Docker: What you need is what you want.
Shared Object images in Docker: What you need is what you want.
 
2009 cluster user training
2009 cluster user training2009 cluster user training
2009 cluster user training
 
Hadoop installation
Hadoop installationHadoop installation
Hadoop installation
 
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
Drupal Camp Brighton 2015: Ansible Drupal Medicine showDrupal Camp Brighton 2015: Ansible Drupal Medicine show
Drupal Camp Brighton 2015: Ansible Drupal Medicine show
 
Ops for everyone - John Britton
Ops for everyone - John BrittonOps for everyone - John Britton
Ops for everyone - John Britton
 
rake puppetexpert:create - Puppet Camp Silicon Valley 2014
rake puppetexpert:create - Puppet Camp Silicon Valley 2014rake puppetexpert:create - Puppet Camp Silicon Valley 2014
rake puppetexpert:create - Puppet Camp Silicon Valley 2014
 
Drupal from scratch
Drupal from scratchDrupal from scratch
Drupal from scratch
 
Medicine show2 Drupal Bristol Camp 2015
Medicine show2 Drupal Bristol Camp 2015Medicine show2 Drupal Bristol Camp 2015
Medicine show2 Drupal Bristol Camp 2015
 
JDD 2017: Nginx + Lua = OpenResty (Marcin Stożek)
JDD 2017: Nginx + Lua = OpenResty (Marcin Stożek)JDD 2017: Nginx + Lua = OpenResty (Marcin Stożek)
JDD 2017: Nginx + Lua = OpenResty (Marcin Stożek)
 
Ansible - Swiss Army Knife Orchestration
Ansible - Swiss Army Knife OrchestrationAnsible - Swiss Army Knife Orchestration
Ansible - Swiss Army Knife Orchestration
 
Ansible, Simplicity, and the Zen of Python
Ansible, Simplicity, and the Zen of PythonAnsible, Simplicity, and the Zen of Python
Ansible, Simplicity, and the Zen of Python
 
Top Node.js Metrics to Watch
Top Node.js Metrics to WatchTop Node.js Metrics to Watch
Top Node.js Metrics to Watch
 
PuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of PuppetPuppetCamp SEA 1 - Use of Puppet
PuppetCamp SEA 1 - Use of Puppet
 
The Puppet Debugging Kit: Building Blocks for Exploration and Problem Solving...
The Puppet Debugging Kit: Building Blocks for Exploration and Problem Solving...The Puppet Debugging Kit: Building Blocks for Exploration and Problem Solving...
The Puppet Debugging Kit: Building Blocks for Exploration and Problem Solving...
 
Puppet Camp Phoenix 2015: Managing Files via Puppet: Let Me Count The Ways (B...
Puppet Camp Phoenix 2015: Managing Files via Puppet: Let Me Count The Ways (B...Puppet Camp Phoenix 2015: Managing Files via Puppet: Let Me Count The Ways (B...
Puppet Camp Phoenix 2015: Managing Files via Puppet: Let Me Count The Ways (B...
 
Docker & FieldAware
Docker & FieldAwareDocker & FieldAware
Docker & FieldAware
 
Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)Configuration surgery with Augeas (OggCamp 12)
Configuration surgery with Augeas (OggCamp 12)
 
AnsibleFest 2014 - Role Tips and Tricks
AnsibleFest 2014 - Role Tips and TricksAnsibleFest 2014 - Role Tips and Tricks
AnsibleFest 2014 - Role Tips and Tricks
 

Destaque

Agile deployment predictive analytics on hadoop
Agile deployment predictive analytics on hadoopAgile deployment predictive analytics on hadoop
Agile deployment predictive analytics on hadoop
DataWorks Summit
 

Destaque (8)

Using docker for data science - part 2
Using docker for data science - part 2Using docker for data science - part 2
Using docker for data science - part 2
 
Using Docker for GPU Accelerated Applications
Using Docker for GPU Accelerated ApplicationsUsing Docker for GPU Accelerated Applications
Using Docker for GPU Accelerated Applications
 
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
 
Deploying Data Science with Docker and AWS
Deploying Data Science with Docker and AWSDeploying Data Science with Docker and AWS
Deploying Data Science with Docker and AWS
 
Running Docker clusters on AWS (November 2016)
Running Docker clusters on AWS (November 2016)Running Docker clusters on AWS (November 2016)
Running Docker clusters on AWS (November 2016)
 
Deep Learning with AWS (November 2016)
Deep Learning with AWS (November 2016)Deep Learning with AWS (November 2016)
Deep Learning with AWS (November 2016)
 
Reproducible bioinformatics pipelines with Docker and Anduril
Reproducible bioinformatics pipelines with Docker and AndurilReproducible bioinformatics pipelines with Docker and Anduril
Reproducible bioinformatics pipelines with Docker and Anduril
 
Agile deployment predictive analytics on hadoop
Agile deployment predictive analytics on hadoopAgile deployment predictive analytics on hadoop
Agile deployment predictive analytics on hadoop
 

Semelhante a Using python and docker for data science

Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018
Ortus Solutions, Corp
 

Semelhante a Using python and docker for data science (20)

Docker workshop DevOpsDays Amsterdam 2014
Docker workshop DevOpsDays Amsterdam 2014Docker workshop DevOpsDays Amsterdam 2014
Docker workshop DevOpsDays Amsterdam 2014
 
Docker - from development to production (PHPNW 2017-09-05)
Docker - from development to production (PHPNW 2017-09-05)Docker - from development to production (PHPNW 2017-09-05)
Docker - from development to production (PHPNW 2017-09-05)
 
Docker for Web Developers: A Sneak Peek
Docker for Web Developers: A Sneak PeekDocker for Web Developers: A Sneak Peek
Docker for Web Developers: A Sneak Peek
 
Into The Box 2018 Going live with commandbox and docker
Into The Box 2018 Going live with commandbox and dockerInto The Box 2018 Going live with commandbox and docker
Into The Box 2018 Going live with commandbox and docker
 
Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018Going live with BommandBox and docker Into The Box 2018
Going live with BommandBox and docker Into The Box 2018
 
Play With Docker
Play With DockerPlay With Docker
Play With Docker
 
Docker workshop
Docker workshopDocker workshop
Docker workshop
 
Running Docker in Development & Production (DevSum 2015)
Running Docker in Development & Production (DevSum 2015)Running Docker in Development & Production (DevSum 2015)
Running Docker in Development & Production (DevSum 2015)
 
From Docker Run To Docker Compose
From Docker Run To Docker ComposeFrom Docker Run To Docker Compose
From Docker Run To Docker Compose
 
Docker tips & tricks
Docker  tips & tricksDocker  tips & tricks
Docker tips & tricks
 
Docker for dev
Docker for devDocker for dev
Docker for dev
 
파이썬 개발환경 구성하기의 끝판왕 - Docker Compose
파이썬 개발환경 구성하기의 끝판왕 - Docker Compose파이썬 개발환경 구성하기의 끝판왕 - Docker Compose
파이썬 개발환경 구성하기의 끝판왕 - Docker Compose
 
Docker Introductory workshop
Docker Introductory workshopDocker Introductory workshop
Docker Introductory workshop
 
Hands-On Session Docker
Hands-On Session DockerHands-On Session Docker
Hands-On Session Docker
 
Create a Database Application Development Environment with Docker
Create a Database Application Development Environment with DockerCreate a Database Application Development Environment with Docker
Create a Database Application Development Environment with Docker
 
Docker by Example - Basics
Docker by Example - Basics Docker by Example - Basics
Docker by Example - Basics
 
2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps
2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps
2017-03-11 02 Денис Нелюбин. Docker & Ansible - лучшие друзья DevOps
 
Docker security
Docker securityDocker security
Docker security
 
Architecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based DeploymentsArchitecting .NET Applications for Docker and Container Based Deployments
Architecting .NET Applications for Docker and Container Based Deployments
 
Lessons from running potentially malicious code inside Docker containers
Lessons from running potentially malicious code inside Docker containersLessons from running potentially malicious code inside Docker containers
Lessons from running potentially malicious code inside Docker containers
 

Último

Último (20)

HR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.comHR Software Buyers Guide in 2024 - HRSoftware.com
HR Software Buyers Guide in 2024 - HRSoftware.com
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
 
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdfThe Top App Development Trends Shaping the Industry in 2024-25 .pdf
The Top App Development Trends Shaping the Industry in 2024-25 .pdf
 
BUS PASS MANGEMENT SYSTEM USING PHP.pptx
BUS PASS MANGEMENT SYSTEM USING PHP.pptxBUS PASS MANGEMENT SYSTEM USING PHP.pptx
BUS PASS MANGEMENT SYSTEM USING PHP.pptx
 
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park %in ivory park+277-882-255-28 abortion pills for sale in ivory park
%in ivory park+277-882-255-28 abortion pills for sale in ivory park
 
Optimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTVOptimizing AI for immediate response in Smart CCTV
Optimizing AI for immediate response in Smart CCTV
 
%in Midrand+277-882-255-28 abortion pills for sale in midrand
%in Midrand+277-882-255-28 abortion pills for sale in midrand%in Midrand+277-882-255-28 abortion pills for sale in midrand
%in Midrand+277-882-255-28 abortion pills for sale in midrand
 
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdfThe Ultimate Test Automation Guide_ Best Practices and Tips.pdf
The Ultimate Test Automation Guide_ Best Practices and Tips.pdf
 
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
 
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
The Real-World Challenges of Medical Device Cybersecurity- Mitigating Vulnera...
 
8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech students8257 interfacing 2 in microprocessor for btech students
8257 interfacing 2 in microprocessor for btech students
 
10 Trends Likely to Shape Enterprise Technology in 2024
10 Trends Likely to Shape Enterprise Technology in 202410 Trends Likely to Shape Enterprise Technology in 2024
10 Trends Likely to Shape Enterprise Technology in 2024
 
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected WorkerHow To Troubleshoot Collaboration Apps for the Modern Connected Worker
How To Troubleshoot Collaboration Apps for the Modern Connected Worker
 
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
 
Right Money Management App For Your Financial Goals
Right Money Management App For Your Financial GoalsRight Money Management App For Your Financial Goals
Right Money Management App For Your Financial Goals
 
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
Shapes for Sharing between Graph Data Spaces - and Epistemic Querying of RDF-...
 
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
Direct Style Effect Systems -The Print[A] Example- A Comprehension AidDirect Style Effect Systems -The Print[A] Example- A Comprehension Aid
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
 
Define the academic and professional writing..pdf
Define the academic and professional writing..pdfDefine the academic and professional writing..pdf
Define the academic and professional writing..pdf
 
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdfPayment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
Payment Gateway Testing Simplified_ A Step-by-Step Guide for Beginners.pdf
 
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain
 

Using python and docker for data science

  • 1. USING DOCKER FOR DATA SCIENCE
  • 2. WHY DOCKER Portable environment Isolated between projects Stateless Fast local file access Hetrogenous
  • 3. GET DOCKER https://docs.docker.com/installation/ boot2docker .dmg or .exe apt-get install docker.io ...
  • 4. RUN SCIPYSERVER $ docker run -d -e "PASSWORD=YourPassword?" ipython/scipyserver $ docker run -d -e "PASSWORD=YourPassword?" --name dev_notebook -p 443:8888 ipython/scipyserver https://localhost:443 https://{boot2docker ip}:443
  • 5. CREATE DATA-ONLY CONTAINERS $ docker run -d -v ~/notebooks:/notebooks --name notebooks_container ubuntu echo notebooks $ docker run -d -v ~/data:/data --name data_container ubuntu echo data
  • 6. MOUNT DATA-ONLY CONTAINERS $ docker stop dev_notebook $ docker rm dev_notebook $ docker run -d -e "PASSWORD=YourPassword?" --name dev_notebook -p 443:8888 --volumes-from data_container --volumes-from notebooks_container ipython/scipyserver
  • 7. CREATE A DOCKERFILE FROM ipython/scipyserver MAINTAINER Calvin Giles <calvin.giles@gmail.com> COPY requirements.txt /requirements.txt RUN pip2 install -r /requirements.txt RUN pip3 install -r /requirements.txt $ docker build -t calvingiles/ds-notebook . $ docker run -d -e "PASSWORD=YourPassword?" --name dev_notebook -p 443:8888 --volumes-from data_container --volumes-from notebooks_container calvingiles/ds-notebook
  • 8. NEXT TIME Connecting to local database containers Tweaking the boot2docker vm memory from 2GB to 8 (or more...) Linking to private git repositories Automated builds with github linking
  • 9. MORE? Docker User Guide: http://docs.docker.com/userguide/ Docker Reference: http://docs.docker.com/reference/commandline/cli/ ipython docker images: https://registry.hub.docker.com/repos/ipython/ my docker image: https://github.com/calvingiles/ds-notebook https://registry.hub.docker.com/u/calvingiles/ds-notebook/
  • 10. ABOUT ME Calvin Giles Data Scientist at Adthena PyData Meetup Organiser untangleconsulting.io calvin.giles@gmail.com @calvingiles on twitter, github, docker hub (and many more)