Developing and experimenting with machine learning models in Python is easy and well supported by robust and agile libraries such as scikit-learn, although efficiently deploying multi-model systems at scale is still a challenge in the data science field. This talk will focus on the main issues related to deploying machine learning models and how to make scikit-learn production-ready with minimal operational efforts, by means of Cloud Computing services, in particular Amazon Web Services. Prerequisites: basic Machine Learning understanding (modeling and training), minimal knowledge about scikit-learn and Python utilities such as Pandas and boto.

1. How to deploy Machine Learning
models in the Cloud
Alex Casalboni
PYCON SETTE
@ Firenze
clda.co/pycon7-alex

2. About Me
PYCON SETTE @ FIRENZE
@alex_casalboni
clda.co/pycon7-alex
Bachelor in Computer Science
Master in Sound & Music Engineering
6+ years of Web Development
3+ years of Python

3. Have you ever met a Data Scien@st?
PYCON SETTE @ FIRENZE
Bob
Bob leads a skilled team of data scienCsts
Bob has built a great ML model
Bob loves Python
Bob likes user-­‐centered ML
clda.co/pycon7-alex

4. How did Bob build his model in Python?
PYCON SETTE @ FIRENZE
clda.co/pycon7-code
clda.co/pycon7-alex
Bob

5. PYCON SETTE @ FIRENZE
GPU availability
distributed compuCng
infrastructure-­‐aware code
1. Bob needs Elas@city
clda.co/pycon7-alex
And why is deploying a challenge?
Bob

6. PYCON SETTE @ FIRENZE
MulC-­‐model architecture
A/B tesCng & versioning
RESTful API
2. Bob needs plenty of Features
Auth layer
clda.co/pycon7-alex
And why is deploying a challenge?
Bob

7. PYCON SETTE @ FIRENZE
infrastructure maintenance
conCnuous delivery & deployments
automaCon, performance & monitoring
3. Data Scien@sts ain’t DevOps
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data
ScienCst
DevOps

8. PYCON SETTE @ FIRENZE
model design, training & tesCng
code reviews & debugging
data exploraCon
4. DevOps ain’t Data Scien@sts
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data
ScienCst
DevOps

9. Deployment Strategies
PYCON SETTE @ FIRENZE
1. Web-­‐app controller
clda.co/pycon7-alex
how will Bob update his model(s)?
same website auth layer?
shared (limited) uWSGI processes?
It’s disarmingly simple, but…
Bob

10. Deployment Strategies
PYCON SETTE @ FIRENZE
2. Fleet of servers
clda.co/pycon7-alex
same problems as before
Bob has many more machines to maintain
sCll no elasCcity (over-­‐provisioning)
Bigger capacity and no code changes, but…
Bob Bob2 Bob3
Load Balancing

11. Deployment Strategies
PYCON SETTE @ FIRENZE
3. Auto Scaling
clda.co/pycon7-alex
<Bob>
sCll shared resources? containers?
even bigger lack of ownership
what about caching, versioning and auth?
Bob achieved elasCcity and availability , but…
AWS ELB + Auto Scaling
(or maybe ElasCc Beanstalk?)

12. Deployment Strategies
PYCON SETTE @ FIRENZE
4. Serverless
clda.co/pycon7-alex
+
Amazon API
Gateway
AWS
Lambda
Happy
Bob
Infinite scalability, global CDN and caching
Transparent elasCcity (and PAYG)
RESTful & auth layer + versioning (for free!)
Bob doesn’t need to be a DevOps anymore!

13. The Serverless Cloud
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
There is always a server somewhere,
you just don't have to worry about it :)

14. Does Bob’s code change a lot?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Basically, no change!
Bob

15. How does Bob keep the model updated?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Bob
Hint: use gzip!

16. PYCON SETTE @ FIRENZE
clda.co/pycon7-lambda
clda.co/pycon7-alex

17. Serverless Cloud Domina@on
PYCON SETTE @ FIRENZE
Amazon Web Services (Lambda)
Microso` Azure (FuncCons)
Google Cloud Plaborm (FuncCons)
IBM Bluemix (OpenWhisk)
hook.io
Cloud PlaSorms Hybrid & Microservices
stackhut.com
webtask.io
iron.io
clda.co/pycon7-alex

18. Thank you :)
jobs.cloudacademy.com
P.S. we are hiring!