This is an introduction to polyaxon and why I use polyaxon. Polyaxon enables me to leverage kubernetes to achieve the objectives: - Make the lead time of experiments as short as possible. - Make the financial cost to train models as cheap as possible. - Make the experiments reproducible.

1. Introduction to Polyaxon
Yu Ishikawa

2. Agenda
- Why do we need Polyaxon?
- What is Polyaxon?
- How does Polyaxon work?
- Demo
- Summary

3. Objectives to introduce Polyaxon
- Make the lead time of experiments as short as possible.
- Make the financial cost to train models as cheap as possible.
- Make the experiments reproducible.
Experiment Phase Operating Phase
Problem
Setting
Collecting
Data
Experiment
s
Off-line
Evaluation
Serving
Models
On-line
Evaluation
Retrain
Model
Off-line
Evaluation
Productionize
ML system
Productionize PhaseML
Workflow
Polyaxon’s role

4. Why do we need Polyaxon?
- We are not able to manage experiments as team today.
- The cost of experiments is expensive in terms of the financial cost and time. There is room to
improve the efficiency and productivity.
- Setting experiment environments can be tough for ML engineers. Moreover, the environments tend
not to be reproducible. Taking over other member’s tasks can be expensive. As well as, we can not
manage the training process as team.
- It takes a long time for python ML libraries like sklearn to do hyperparameter search, since python is
basically not good at scalability.

5. Agenda
- Why do we need Polyaxon?
- What is Poyaxon?
- How does Polyaxon work?
- Demo
- Summary

6. What is Polyaxon?
- An open source platform for reproducible machine learning at scale.
- https://polyaxon.com/
- Features
- Notebook
- Hyperparameter search
- Powerful workspace
- User management
- Dynamic resources allocation
- Dashboard
- Versioning

7. Notebook environment with Jupyter
---
version: 1
kind: notebook
build:
image: tensorflow/tensorflow:1.4.1-py3
build_steps:
- pip3 install jupyter
$ Polyaxon notebook start -f polyaxon_notebook.yml
New version of CLI (0.3.5) is now available. To upgrade run:
pip install -U polyaxon-cli
Notebook is being deployed for project `quick-start`
It may take some time before you can access the notebook.
Your notebook will be available on:
http://35.184.217.84:80/notebook/root/quick-start/
- Polyaxon enables us to launch a jupyter environment with one command. As
well as we can define the environment with docker and some commands in a
YAML file.
- We can reproduce the notebook experiments easily.

8. Hyperparameter tuning with Polyaxon
- Polyaxon supports some hyperparameter tuning methods:
- Grid search
- Random search
- Bayesian optimization
- Early stopping
- Hyperband
- We can control the concurrency of hyperparameter tuning with YAML file.
- We can reproduce hyperparameter tuning jobs as well.

9. Hyperparameter tuning with high concurrency
---
version: 1
kind: group
hptuning:
concurrency: 5
matrix:
learning_rate:
linspace: 0.001:0.1:5
dropout:
values: [0.25, 0.3]
activation:
values: [relu, sigmoid]
declarations:
batch_size: 128
num_steps: 500
num_epochs: 1
build:
image: tensorflow/tensorflow:1.4.1-py3
build_steps:
- pip3 install --no-cache-dir -U polyaxon-helper
run:
cmd: python3 model.py --batch_size={{ batch_size }}
--num_steps={{ num_steps }}
--learning_rate={{ learning_rate }}
--dropout={{ dropout }}
--num_epochs={{ num_epochs }}
--activation={{ activation }}
$ Polyaxon notebook start -f
polyaxon_notebook.yml
New version of CLI (0.3.5) is now available. To
upgrade run:
pip install -U polyaxon-cli
Creating an experiment group with the following
definition:
---------------- -----------------
Search algorithm grid
Concurrency 5 concurrent runs
Early stopping deactivated
---------------- -----------------
Experiment group 1 was created
polyaxon_gridsearch.yml

10. Dashboard ~ Experiments

11. Dashboard ~ Metrics Visualization

12. Agenda
- Why do we need Polyaxon?
- What is Polyaxon?
- How does Polyaxon work?
- Demo
- Summary

13. Hyperparameter tuning with a single machine
Many CPU cores machine
Memory
data
CPU
Core: Train with parameter set A
Core: Train with parameter set B
Core: Train with parameter set C
Core: Train with parameter set X
...
For instance, scikit-learn’s GridSearchCV enables us to run experiments in parallel. However, the number
of process is based on the number of CPU cores. For instance, a 64 CPU cores machine can have up to
64 concurrencies.

14. Hyperparameter tuning of Polyaxon
Polyaxon on k8s
Polyaxon core
Node A
Node B
Node C
Training Code
Upload & run
Build
Pod: parameter set A
Pod: parameter set B
Pod: parameter set C
Pod: parameter set D
Pod: parameter set E
Pod: parameter set F
Schedule
The more the number of nodes in k8s cluster, the more the number of process to
train is. There is no constraint of parallelism.

15. Even one experiments can be shorter
Experiments Evaluation
Experiments Evaluation
By leveraging the multiple nodes on k8s, we can shorten the experiment time of 1
experiment with high concurrency.
Single machine
Polyaxon
t
Reduce training time

16. Auto-scalable & preemptible node pool with polyaxon
Polyaxon on k8s
Node pool for
polyaxon core
Node Node Node
Node pool for
experiments

17. Auto-scalable & preemptible node pool with polyaxon
Polyaxon on k8s
Node pool for
polyaxon core
Node Node Node
Node pool for
experiments
Training Code
of experiment X
Upload & run
Concurrency:
- 100
Requests:
- CPU: 1
- Memory: 2GB

18. Auto-scalable & preemptible node pool with polyaxon
Polyaxon on k8s
Node pool for
polyaxon core
Node Node Node
Node pool for
experiments
Preemptible node Preemptible node Preemptible node
Training Code
of experiment X
Upload & run
Concurrency:
- 100
Requests:
- CPU: 1
- Memory: 2GB
Automatically launch new preemptible instances

19. Auto-scalable & preemptible node pool with polyaxon
Polyaxon on k8s
Node pool for
polyaxon core
Node Node Node
Node pool for
experiments
Training Code
of experiment Y
Upload & run
Concurrency:
- 50
Requests:
- CPU: 1
- Memory: 1GB
Preemptible node Preemptible node Preemptible node

20. Auto-scalable & preemptible node pool with polyaxon
Polyaxon on k8s
Node pool for
polyaxon core
Node Node Node
Node pool for
experiments
Preemptible node Preemptible node
Preemptible node Preemptible node
Preemptible node
Training Code
of experiment Y
Upload & run
Concurrency:
- 50
Requests:
- CPU: 1
- Memory: 1GB

21. Preemptible instance/GPU/TPU pricing
Preemptible instance Preemptible GPU
Preemptible TPU

22. Regular instance cost vs Preemptible instance cost
Running cost
t
Regular instance
Preemptible instance
- We can reduce the cost of training models by
leveraging preemptible instances with
polyaxon.
- Polyaxon enables us to use preemptible node
pool for experiments.
- Since polyaxon automatically scale the node
pool with GKE, we don’t need to hold static
instances for experiments.
Reduced
cost

23. It takes a longer time to do experiments with a single machine sequentially,
because python ML library like sklearn is not basically scalable.
Multiple Experiments with a single machine
Experiments Evaluation
Experiments Evaluation
Experiments Evaluation
t

24. Polyaxon enables us to easily run multiple experiments in parallel on k8s. We
don’t need to wait for each experiments to move on to the next one.
Multiple Experiments with polyaxon
Experiments Evaluation
Experiments Evaluation
Experiments Evaluation
We can shorten the total experiments time by
the parallelism of Polyaxon.
t

25. Sequential experiments cost vs parallel experiments cost
Running cost
t
- Essentially speaking, the costs of instances
should be the same, since the cost of CPU
usage is linear with running time.
- However, we should not overlook labor costs
while experiments. Waiting for experiments is
time and money wasting. Time is money!!
- We can reduce the total cost by shortening the
total experiments time.
Sequential experiments
Experiments in parallel
Labor cost
t
Reduced cost

26. Power of multiple preemptible nodes
- The cost of preemptible n1-standard-64 x 10 nodes x 2 hours with 640
concurrencies. It should be cheap!
- $12.8 = $0.64 * 10 * 2
- Even If it takes about 20 minutes to run 1 parameter set of a training job, we
can run about 3840 parameter sets of a training job for just 2 hours with such
a cheap cost.
We can achieve the objectives:
- Make the lead time of experiments as short as possible.
- Make the financial cost to train models as cheap as possible.

27. Agenda
- Why do we need Polyaxon?
- What is Polyaxon?
- How does Polyaxon work?
- Demo
- Summary

28. Demo
- Notebook
- Job
- Experiment
- Hyperparameter tuning at scale

29. Summary
- We can definitely achieve the objectives with polyaxon on GKE.
- Make the lead time of experiments as short as possible.
- Make the financial cost to train models as cheap as possible.
- Make the experiments reproducible.
- All we ML engineers have to do is:
- Making the training code in python as usual, and
- Defining the YAML files to do experiments.
- What’s next?
- Supporting preemptible GPUs / TPUs.

30. Appendix A: Links
- Polyaxon
- https://polyaxon.com/
- Documentation
- https://docs.polyaxon.com/
- Examples
- https://github.com/polyaxon/polyaxon-quick-start
- https://github.com/polyaxon/deep-learning-with-python-notebooks-on-polyaxon
- https://github.com/polyaxon/polyaxon-examples

31. Appendix B: Architecture of Polyaxon