How to use the Data Science Virtual Machine to do Deep Learning in the cloud

1. Azure and Deep Learning

2. • David Giard
• Senior Microsoft Technical Evangelist
• dgiard@Microsoft.com
• Davidgiard.com
• @davidgiard

3. Cloud Computing
Host some or all of your data or application
on a third-party server
in a highly-scalable, highly-reliable way

4. Cloud Service
IAAS Infrastructure as a Service
PAAS Platform as a Service
SAAS Software as a Service

5. Virtualization
Server1.vhd

6. Virtualization
Server1.vhd Server2.vhd Server3.vhd Server4.vhd

7. Cloud Services
IaaS
Managedbyvendor
Storage
Servers
Networking
O/S
Middleware
Virtualization
Data
Applications
Runtime
Youscale,make
resilient&manage
PaaS
Scale,resilienceand
managementbyvendor
Youmanage
Storage
Servers
Networking
O/S
Middleware
Virtualization
Applications
Runtime
Data
On Premises
Youscale,makeresilientandmanage
Storage
Servers
Networking
O/S
Middleware
Virtualization
Data
Applications
Runtime
SaaS
Storage
Servers
Networking
O/S
Middleware
Virtualization
Applications
Runtime
Data
Scale,resilienceand
managementbyvendor
More Control
More Work
Less Control
Less Work

8. Reliability

9. Reliability
Data or ApplicationData or ApplicationData or Application

10. Elasticity

11. Elasticity
0
1
2
3
4
5
6
7
8
9
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

12. Elasticity
0
1
2
3
4
5
6
7
8
9
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

13. Elasticity
0
1
2
3
4
5
6
7
8
9
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

14. Elasticity
0
1
2
3
4
5
6
7
8
9
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

15. Elasticity
0
1
2
3
4
5
6
7
8
9
Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri

16. Elasticity
0
1
2
3
4
5
6
7
8
9
1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00

17. Elasticity
0
1
2
3
4
5
6
7
8
9
1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00

18. Azure

19. Portal

20. Creating a VM

22. Scripting
• PowerShell
• CLI

23. ARM Template

24. PowerShell
# Login to Azure
Login-AzureRmAccount
# Create a Resource Group
New-AzureRmResourceGroup -Name dgtestrg -Location "South Central US"
# Deploy from an ARM template
New-AzureRmResourceGroupDeployment `
-ResourceGroupName dgtestrg `
-TemplateFile template.json `
-TemplateParameterFile .parameters.json

25. CLI
# Log into Azure
az login -u <username> -p <password>
# Create a resource group
az group create --name dgtestclirg --location southcentralus
# Deploy from ARM template
az group deployment create
--name dgtestclivm
--resource-group dgtestclirg
--template-file template.json
--parameters parameters.json

26. Data Science Virtual Machine for Linux
(Ubuntu)
• TensorFlow
• Microsoft Cognitive Toolkit
• MXNet
• Caffe
• Caffe2
• Chainer
• NVIDIA DIGITS
• Deep Water
• Keras
• Theano
• Torch
• PyTorch

27. Data Science Virtual Machine for Linux
(Ubuntu)
• Microsoft R Server 9.2.1 with Microsoft R Open 3.4.1, MicrosoftML package with machine learning algorithms, RevoScaleR and
revoscalepy for distributed and remote computing, and R and Python Operationalization
• Anaconda Python 2.7 and 3.5
• JupyterHub with sample notebooks
• Spark local 2.2.0 with PySpark and SparkR Jupyter kernels
• Single node local Hadoop
• Azure command-line interface
• Visual Studio Code, IntelliJ IDEA, PyCharm, and Atom
• H2O, Deep Water, and Sparkling Water
• Julia
• Vowpal Wabbit for online learning
• xgboost for gradient boosting
• SQL Server 2017
• Intel Math Kernel Library

28. Connect
• bash / ssh
• PuTTY (https://www.putty.org)
• x2Go (https://wiki.x2go.org/doku.php)

29. GPU
• NC-Series Azure VMs
• Not available in every region
(https://azure.microsoft.com/en-us/regions/services/)
• Select VM Image with GPU driver installed (e.g., Data Science VM)
• Select HDD; Not SDD
• Need to be “activated” for some accounts. Open a Quota support
ticket.

30. Recommendations
• Script resource management
• Shut down VMs when not in use
• Set Auto-Shutdown time

31. Demo

32. Links
• From https://blogs.msdn.microsoft.com/sarahsays/2018/01/04/whirlwind-
tour-of-recurrent-neural-networks-prerequisites/:
• Step 6: git clone https://github.com/jcjohnson/torch-rnn.git and torch-hdf5 stuff
• Follow https://blogs.msdn.microsoft.com/sarahsays/2018/01/04/a-
whirlwind-tour-of-recurrent-neural-networks-the-hard-way/
• Intro to Deep Learning VM: https://docs.microsoft.com/en-
us/azure/machine-learning/data-science-virtual-machine/deep-learning-
dsvm-overview
• Create a Deep Learning VM: https://docs.microsoft.com/en-
us/azure/machine-learning/data-science-virtual-machine/provision-deep-
learning-dsvm
• My blog: http://davidgiard.com