If you want to deploy Machine Learning models into production, you have to be very careful in order to not break the running system. This talk shows one simple approach to make sure that the new models don't break your production system.

1. 1Digital Product School by UnternehmerTUM

2. 2Digital Product School by UnternehmerTUM

3. 3Digital Product School by UnternehmerTUM
Classification in Prod!

4. 4Digital Product School by UnternehmerTUM
Recommender in Prod!

5. 5Digital Product School by UnternehmerTUM
Chatbot in Prod!

6. Ship your Machine Learning Application
with Confidence

7. 7Digital Product School by UnternehmerTUM
About Michi
Michael Stockerl
Head of Software Engineering - UnternehmerTUM
stockerl@unternehmertum.de
Before:
• Data Engineer @ gutefrage.net
• Data Engineer @ Gini
• M.Sc. Computer Science @LMU

8. 8Digital Product School by UnternehmerTUM
The Bavarian Stanford

9. 9Digital Product School by UnternehmerTUM
About UnternehmerTUM
2 of our Programs:
4 of our Start-Ups:
4 of our Partners:

10. 10Digital Product School by UnternehmerTUM
Who is building a ML App?
Example: Detect Ships
SHIP
SHIP

11. 11Digital Product School by UnternehmerTUM
What is the Team?

12. 12Digital Product School by UnternehmerTUM
App Developer

13. 13Digital Product School by UnternehmerTUM
Data Engineer

14. 14Digital Product School by UnternehmerTUM
Data Scientist

15. 15Digital Product School by UnternehmerTUM
How to release a new model?

16. 16Digital Product School by UnternehmerTUM
Data Scientist
Push Model
Continuous Delivery Pipeline

17. 17Digital Product School by UnternehmerTUM
Data Scientist
Push Model
Continuous Delivery Pipeline
Trigger CD
Unit Test &
Build

18. 18Digital Product School by UnternehmerTUM
Data Scientist
Push Model
Continuous Delivery Pipeline
Trigger CD
Unit Test &
Build
Pull Test Set
Test Set
Validation

19. 19Digital Product School by UnternehmerTUM
Data Scientist
Push Model
Continuous Delivery Pipeline
Trigger CD
Unit Test &
Build
Pull Test Set
Test Set
Validation
Deploy
Canary Service

20. 20Digital Product School by UnternehmerTUM
Data Scientist
Push Model
Continuous Delivery Pipeline
Trigger CD
Unit Test &
Build
Pull Test Set
Test Set
Validation
Deploy
Canary Service
MonitoringPerformance Metrics
* p99 Response Time
* Error Rate

21. 21Digital Product School by UnternehmerTUM
Data Scientist
Push Model
Continuous Delivery Pipeline
Trigger CD
Unit Test &
Build
Pull Test Set
Test Set
Validation
Deploy
Canary Service
MonitoringPerformance Metrics
* p99 Response Time
* Error Rate
Request
Shadowing

22. 22Digital Product School by UnternehmerTUM
Test the Model in the Real World

23. 23Digital Product School by UnternehmerTUM
Request Shadowing Close Up
classification
incoming_pics
canary_classification

24. 24Digital Product School by UnternehmerTUM
Request Shadowing Close Up
classification
incoming_pics
canary_classification
Classification Result

25. 25Digital Product School by UnternehmerTUM
Request Shadowing Close Up
classification
incoming_pics
canary_classification
joined_classification
Find differences

26. 26Digital Product School by UnternehmerTUM
Spot Checking
Select wrong classifications!
Yes
Boat
No Yes No
Yes No Yes No

27. 27Digital Product School by UnternehmerTUM
Build up Test Data

28. 28Digital Product School by UnternehmerTUM
There is no AI without Engineering!

29. Questions?

30. If you want to solve real world problems
Apply at https://dpschool.io

31. 31Digital Product School by UnternehmerTUM
References
Pictures:
• https://de.wikipedia.org
• https://www.gutefrage.net
• https://www.twitter.com
• Modelsmaker.org
• pxhere.com