Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2mBDLpb. Corey Zumar offers an overview of MLflow – a new open source platform to simplify the machine learning lifecycle from Databricks. MLflow provides APIs for tracking experiment runs between multiple users within a reproducible environment and for managing the deployment of models to production. MLflow is designed to be an open, modular platform. Filmed at qconnewyork.com. Corey Zumar is a software engineer at Databricks, where he’s working on machine learning infrastructure and APIs for the machine learning lifecycle, including model management and production deployment. He is also an active developer of MLflow.

1. A platform for the
Complete Machine
Learning Lifecycle
Corey Zumar
June 24th, 2019

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
mlflow-databricks/

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. 2
Outline
• Overview of ML development challenges
• How MLflow tackles these challenges
• MLflow components
• Demo
• How to get started

5. 3
Machine Learning
Development is Complex

6. 4
ML Lifecycle
Data Prep
Training
Deploy
Raw Data
μ
λ θ Tunin
g
Scal
e
μ
λ θ Tunin
g
Scal
e
Scal
e
Scal
e
Model
Exchang
e
Governanc
e
Delta

7. 5
Custom ML Platforms
Facebook FBLearner, Uber Michelangelo, Google TFX
+Standardize the data prep / training / deploy loop:
if you work with the platform, you get these!
–Limited to a few algorithms or frameworks
–Tied to one company’s infrastructure
Can we provide similar benefits in an open
manner?

8. 6
Introducing
Open machine learning platform
• Works with any ML library & language
• Runs the same way anywhere (e.g. any cloud)
• Designed to be useful for 1 or 1000+ person orgs

9. 7
MLflow Components
Tracking
Record and query
experiments: code,
configs, results,
…etc
Project
s
Packaging format
for reproducible
runs on any
platform
Model
s
General model
format
that supports diverse
deployment tools

10. 8
Key Concepts in Tracking
Parameters: key-value inputs to your code
Metrics: numeric values (can update over time)
Source: training code that ran
Version: version of the training code
Artifacts: files, including data and models
Tags and Notes: any additional information

11. 9
MLflow Tracking
Tracking Server
UI
API
Tracking APIs
(REST, Python, Java, R)

12. 10
MLflow Tracking
Tracking
Record and query
experiments: code,
configs, results,
…etc
import mlflow
with mlflow.start_run():
mlflow.log_param("layers", layers)
mlflow.log_param("alpha", alpha)
# train model
mlflow.log_metric("mse", model.mse())
mlflow.log_artifact("plot", model.plot(test_df))
mlflow.tensorflow.log_model(model)

13. 11
Demo
Goal: Classify hand-drawn digits
1. Instrument Keras training code with MLflow tracking APIs
2. Run training code as an MLflow Project
3. Deploy an MLflow Model for real-time serving

14. 12
MLflow backend stores
1. Entity (Metadata) Store
• FileStore (local filesystem)
• SQLStore (via SQLAlchemy)
• REST Store
2. Artifact Store
• S3 backed store
• Azure Blob storage
• Google Cloud storage
• DBFS artifact repo

15. 13
MLflow Components
Tracking
Record and query
experiments: code,
configs, results,
…etc
Project
s
Packaging format
for reproducible
runs on any
platform
Model
s
General model
format
that supports diverse
deployment tools

16. 14
MLflow Projects Motivation
Diverse set of training
tools
Diverse set of
environments
Challenge:
ML results are
difficult to
reproduce.

17. 15
Project Spec
Code
Data
Config
Local Execution
Remote Execution
MLflow Projects
Dependencie
s

18. 16
MLflow Projects
Packaging format for reproducible ML runs
• Any code folder or GitHub repository
• Optional MLproject file with project configuration
Defines dependencies for reproducibility
• Conda (+ R, Docker, …) dependencies can be specified in MLproject
• Reproducible in (almost) any environment
Execution API for running projects
• CLI / Python / R / Java
• Supports local and remote execution

19. 17
Example MLflow Project
my_project/
├── MLproject
│
│
│
│
│
├── conda.yaml
├── main.py
└── model.py
...
conda_env: conda.yaml
entry_points:
main:
parameters:
training_data: path
lambda: {type: float, default: 0.1}
command: python main.py {training_data}
{lambda}
$ mlflow run git://<my_project>

20. 18
Demo
Goal: Classify hand-drawn digits
1. Instrument Keras training code with MLflow tracking APIs
2. Run training code as an MLflow Project
3. Deploy an MLflow Model for real-time serving

21. 19
MLflow Components
Tracking
Record and query
experiments: code,
configs, results,
…etc
Project
s
Packaging format
for reproducible
runs on any
platform
Model
s
General model
format
that supports diverse
deployment tools

22. 20
Inference
Code
Batch & Stream
Scoring
Serving Tools
Mlflow Models Motivation
ML
Frameworks

23. 21
Model Format
Flavor 2Flavor 1
ML
Frameworks
Inference Code
Batch & Stream
Scoring
Serving Tools
Standard for ML
models
MLflow Models

24. 22
MLflow Models
Packaging format for ML Models
• Any directory with MLmodel file
Defines dependencies for reproducibility
• Conda environment can be specified in MLmodel configuration
Model creation utilities
• Save models from any framework in MLflow format
Deployment APIs
• CLI / Python / R / Java

25. 23
Example MLflow Model
my_model/
├── MLmodel
│
│
│
│
│
└ estimator/
├─ saved_model.pb
└─ variables/
...
Usable with Tensorflow
tools / APIs
Usable with any Python
tool
run_id: 769915006efd4c4bbd662461
time_created: 2018-06-28T12:34
flavors:
tensorflow:
saved_model_dir: estimator
signature_def_key: predict
python_function:
loader_module:
mlflow.tensorflow
mlflow.tensorflow.log_model(...)

26. 24
Model Flavors Example
Train a model
mlflow.keras.log_model()
Model
Format
Flavor 1:
Pyfunc
Flavor 2:
Keras
predict = mlflow.pyfunc.load_pyfunc(…)
predict(input_dataframe)
model = mlflow.keras.load_model(…)
model.predict(keras.Input(…))

27. 25
Model Flavors Example
predict = mlflow.pyfunc.load_pyfunc(…)
predict(input_dataframe)

28. 26
Demo
Goal: Classify hand-drawn digits
1. Instrument Keras training code with MLflow tracking APIs
2. Run training code as an MLflow Project
3. Deploy an MLflow Model for real-time serving

29. 27
1.0 Release
MLflow 1.0 was released recently! Major features:
•New metrics UI
•“Step” axis for metrics
•Improved search capabilities
•Package MLflow Models as Docker containers
•Support for ONNX models
27

30. 28
Ongoing MLflow Roadmap
• New component: Model Registry for model management
• Multi-step project workflows
• Fluent Tracking API for Java and Scala
• Packaging projects with build steps
• Better environment isolation when loading models
• Improved model input/output schemas

31. 29
Get started with MLflow
pip install mlflow to get started
Find docs & examples at mlflow.org
tinyurl.com/mlflow-slack
29

32. 30
Thank you!

33. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
mlflow-databricks/