A talk about applying Continuous Delivery to Machine Learning (CD4ML) presented by Arif Wider from ThoughtWorks at NDC Sydney Conference 2019. Abstract: It is already challenging to transition a machine learning model or AI system from the research space to production, and maintaining that system alongside ever-changing data is an even greater challenge. In software engineering, Continuous Delivery practices have been developed to ensure that developers can adapt, maintain, and update software and systems cheaply and quickly, enabling release cycles on the scale of hours or days instead of weeks or months. Nevertheless, in the data science world Continuous Delivery is rarely been applied holistically. This is partly due to different workflows: data scientists regularly work on whole sets of hypotheses, whereas software engineers work more linearly even when evaluating multiple implementation alternatives. Therefore, existing software engineering practices cannot be applied as-is to machine learning projects. Learn how we used our expertise in both fields to adapt practices and tools to allow for Continuous Intelligence–the practice of delivering AI applications continuously.

1. 1
Continuous
Intelligence
Applying Continuous Delivery to
Machine Learning
Arif Wider - ThoughtWorks Germany
NDC Sydney, October 16, 2019
©ThoughtWorks 2019

2. 5000+ technologists with 40 offices in 14 countries
Partner for technology driven business transformation
©ThoughtWorks 2019
join.thoughtworks.comSydney - Melbourne - Brisbane

3. #1
in Agile and
Continuous Delivery
100+
books written
©ThoughtWorks 2019

4. ©ThoughtWorks 2019

5. TECHNIQUES
Continuous delivery
for machine
learning (CD4ML)
models
#8
TRIAL
8
©ThoughtWorks 2019

6. TECHNIQUES
Continuous delivery
for machine
learning (CD4ML)
models
#8
TRIAL
8
©ThoughtWorks 2019

7. 7
WHERE
IT ALL
STARTED
©ThoughtWorks 2019

8. 8
Data Scientists
Developers

9. 9
Data Scientists
Developers
prediction
model
Feedback,
requests
SOUNDS LIKE DEV/OPS?

10. 10
Data Scientists
Developers

11. 11
LET’S DO
CONTINUOUS
DELIVERY!

12. CONTINUOUS DELIVERY PIPELINES
12
Prediction Model Pipeline
Web Application Pipeline

13. 13
Prediction Model Pipeline
Web Application Pipeline
AI4U Data Science, Delivered Continuously – A. Wider & C. Windheuser
CONTINUOUS DELIVERY PIPELINES

14. NEW CHALLENGES
14

15. NEW CHALLENGES
15
→ How to version control training data?
→ Training data and prediction models don’t fit into Git :-(
→ Model re-training slows down the entire continuous delivery server
→ Data scientists want to evaluate several solutions at the same time...
→ ...and they use analytics notebooks which are hard to version control!
→ How to unit test data science code that is tied to changing data?
→ How to prevent behaviour changes of the model to break the application?

16. WHAT ARE THE REASONS?
16

17. DIFFERENT WORKFLOWS
17
master
DEVELOPERS

18. DIFFERENT WORKFLOWS
18
master
change-max-depth
try-random-forest
©ThoughtWorks 2019
DATA SCIENTISTS

19. Continuous Delivery is the ability to get
changes of all types — including new features,
configuration changes, bug fixes and
experiments — into production, or into the
hands of users, safely and quickly in a
sustainable way.
Jez Humble & Dave Farley

20. MORE TYPES OF CHANGE
20
ModelData Code
+ +
Features
Schema
Volume
...
Code
Model Description
Parameters
Hyper-Parameters
...
Portability
Scalability
Maintainability
...
Icons created by Noura Mbarki and I Putu Kharismayadi from Noun Project
©ThoughtWorks 2019

21. MORE TYPES OF PIPELINES
DATA PIPELINE
ML PIPELINE DEPLOYMENT
PIPELINE

22. PUTTING EVERYTHING TOGETHER
22
Data Science,
Model
Building
Training Data
Source Code
+
Executables
Model
Evaluation
Productionize
Model
Integration
Testing
Deployment
Test Data
Model +
parameters
CD Tools and Repositories
DiscoverableandAccessibleData
Monitoring
©ThoughtWorks 2019
Production Data

23. WHAT DO WE NEED IN OUR STACK?
23
Doing CD with Machine Learning is still a hard problem
MODEL
PERFORMANCE
ASSESSMENT
VERSION
CONTROL AND
ARTIFACT
REPOSITORIES
©ThoughtWorks 2019
MONITORING
AND
OBSERVABILITY
DISCOVERABLE
AND
ACCESSIBLE
DATA
CONTINUOUS
DELIVERY
ORCHESTRATION
TO COMBINE
PIPELINES
INFRASTRUCTURE
FOR MULTIPLE
ENVIRONMENTS
AND
EXPERIMENTS

24. WHAT DO WE NEED IN OUR STACK?
24
Doing CD with Machine Learning is still a hard problem
MODEL
PERFORMANCE
ASSESSMENT
VERSION
CONTROL AND
ARTIFACT
REPOSITORIES
©ThoughtWorks 2019
MONITORING
AND
OBSERVABILITY
DISCOVERABLE
AND
ACCESSIBLE
DATA
CONTINUOUS
DELIVERY
ORCHESTRATION
TO COMBINE
PIPELINES
INFRASTRUCTURE
FOR MULTIPLE
ENVIRONMENTS
AND
EXPERIMENTS

25. WHAT DO WE NEED IN OUR STACK?
25
Doing CD with Machine Learning is still a hard problem
MODEL
PERFORMANCE
ASSESSMENT
VERSION
CONTROL AND
ARTIFACT
REPOSITORIES
©ThoughtWorks 2019
MONITORING
AND
OBSERVABILITY
DISCOVERABLE
AND
ACCESSIBLE
DATA
CONTINUOUS
DELIVERY
ORCHESTRATION
TO COMBINE
PIPELINES
INFRASTRUCTURE
FOR MULTIPLE
ENVIRONMENTS
AND
EXPERIMENTS
DEMO

26. DEMO
26
Data Scientist
Develops ML Model
Test and productionize
the model
Deploy to production
servers©ThoughtWorks 2019
Application

27. ©ThoughtWorks 2018 Commercial in Confidence 27

28. 2828
THANK YOU!
Arif Wider
awider@thoughtworks.com
©ThoughtWorks 2019
join.thoughtworks.com