Kubernetes has become the defacto standard as a platform for container orchestration. Its ease of extending and many integrations has paved the way for a wide variety of data science and research tooling to be built on top of it. From all encompassing tools like Kubeflow that make it easy for researchers to build end-to-end Machine Learning pipelines to specific orchestration of analytics engines such as Spark; Kubernetes has made the deployment and management of these things easy. This presentation will showcase some of the larger research tools in the ecosystem and go into how Kubernetes has enabled this easy form of application management.

1. Kubernetes
The Next Research Platform

2. $ whoami
Bob Killen
rkillen@umich.edu
Senior Research Cloud Administrator
CNCF Ambassador
GitHub: @mrbobbytables
Twitter: @mrbobbytables

3. Kubernetes TL;DR Edition
● Greek for “Pilot” or “Helmsman of a ship”.
● Container orchestration system originally developed at Google.
● Built with lessons learned from Borg and Omega.
● Designed from the ground-up as a loosely coupled collection of components
centered around deploying, maintaining and scaling workloads.
● Supports both on-prem and cloud provider deployments.

4. Kubernetes TL;DR Edition
● Declarative system.
● Steers cluster towards desired
state.
● EVERYTHING is an API Object.
● Objects generally describe in
YAML.
apiVersion: batch/v1
kind: Job
metadata:
name: job-example
spec:
backoffLimit: 4
completions: 4
parallelism: 2
template:
spec:
containers:
- name: hello
image: alpine:latest
command: ["/bin/sh", "-c"]
args: ["echo hello from $HOSTNAME!"]
restartPolicy: Never

5. Why?

6. Research needs
are changing.

7. Why?
● Increased use of containers...everywhere.
● Moving away from strict “job” style workflows.
● Adoption of data-streaming and in-flight processing.
● Greater use of interactive Science Gateways.
● Dependence on other more persistent services.

8. Why Kubernetes?
● Kubernetes is seeing significant adoption across
Enterprises and multiple fields of research; serving as
both a scientific platform and substrate for application
management.
● Very large, active development community.
● Extremely easy to extend, augment, and integrate with
other systems.

9. Why Kubernetes?
Use the SAME API
across bare metal
and EVERY cloud
provider.

10. Challenges
● Difficult to integrate with classic multi-user posix
infrastructure.
○ Translating API level identity to posix identity.
● Installation on-prem/bare-metal is not as well supported.
● Device support and integration is a pain point.
○ GPUs well supported, other devices -- not as much.

11. Challenges with Regard to HPC
● Difficult to integrate with classic multi-user posix
infrastructure.
○ Translating API level identity to posix identity.
● No “native” concept of job queue or wall time.
○ Up to higher level components to extend and add that functionality.
● Scheduler generally not as expressive as common HPC
workload managers such as Slurm or Torque.

12. Challenges
Very high learning curve
coming from a traditional
infrastructure background.

13. Ecosystem

14. Helm
https://helm.sh

15. Helm
● “Package manager” for Kubernetes.
○ User only have to configures a few variables for their site without needing
to know majority of details of the application.
● Many commonly used applications packaged and
distributed as “Helm Charts”.

16. List of Charts
● Aerospike
● Airflow
● Argo
● CockroachDB
● Dask
● Flink
● Hadoop
● Galaxy
● Hazelcast
● Ignite
● Jenkins
● JanusGraph
● JupyterHub
● Kafka
● KubeDB
● Luigi
● MariaDB
● Metabase
● MongoDB
● Moodle
● NATS
● Pachyderm
● Postgres
● Presto
● Pulsar
● RECAST
● RabbitMQ
● Spark
● Tensorflow
● Terracotta
● Zookeeper

17. Controllers &
Custom Resources
https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/

18. Controllers & Custom Resources
● Custom Resource Definition (CRD).
● Extends current Kubernetes resources.
● Create your own Kubernetes API object that can be
consumed in the SAME WAY with the SAME TOOLS as
every other Kubernetes object.
● Add custom behaviors to workload management.

19. Example: CRD
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: foo.bar.example.com
spec:
group: bar.example.com
version: v1alpha1
scope: Namespaced
names:
plural: foos
singular: foo
kind: Foo
validation:
openAPIV3Schema:
properties:
spec:
properties:
varFoo:
type: string
apiVersion: foo.bar.example.com/v1alpha1
kind: Foo
metadata:
name: myfoo
spec:
varFoo: bar

20. Example: Kube-batch
● Controller that adds coscheduling (gang scheduling) in the form of a
PodGroup object and additional scheduler.
● Developed by Huawei & IBM.
● Job Queues on the Road Map.
https://github.com/kubernetes-sigs/kube-batch
apiVersion: scheduling.incubator.k8s.io/v1alpha1
kind: PodGroup
metadata:
name: MPIGroup
spec:
minMember: 6

21. Example: Argo
● Powerful suite of workflow tools.
● Workflow engine supports both DAG and
Pipeline based workflows.
● Built-in Event system.
● Integrated and used by many other organizations and
projects.

22. Native vs CRD
apiVersion: batch/v1
kind: Job
metadata:
name: hello-world
spec:
completions: 1
template:
spec:
containers:
- name: hello
image: alpine:latest
command: ["/bin/sh", "-c"]
args: ["echo Hello World”]
restartPolicy: Never
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: hello-world-
spec:
entrypoint: hello
arguments:
parameters:
- name: message
value: Hello World
templates:
- name: hello
inputs:
parameters:
- name: message
container:
image: alpine:latest
command: ["/bin/sh", "-c"]
args: ["echo {{inputs.parameters.message}}"]

23. Operators
https://www.operatorhub.io/what-is-an-operator

24. Operators

25. Operator Pattern
● Uses Controllers & CRDs to manage complex applications.
● Introduced by CoreOS in 2016.
● Automatically handle full application lifecycle: Install,
Configuration, Upgrade, Backup, Failover and Scaling.
● Multiple frameworks available supporting a wide range of
languages and components.

26. Example: Spark
● Kubernetes supported as an executor in 2.3+
● Spark maintainers pursued developing their own controller
as Spark workload patterns did not fit with out of the box
Kubernetes core workload types.
● Bypasses “default” Spark job submission process and
uses a SparkApplication CRD.
https://github.com/GoogleCloudPlatform/spark-on-k8s-operator

27. Example job
apiVersion: sparkoperator.k8s.io/v1beta1
kind: SparkApplication
metadata:
name: myspark
spec:
type: Scala
mode: cluster
image: gcr.io/spark-operator/spark:v2.4.0
mainClass: org.apache.spark.examples.SparkPi
mainApplicationFile: local://spark-example.jar
sparkVersion: 2.4.0
volumes:
- name: test-volume
hostPath:
path: "/tmp"
type: Directory
<continued>
<continued>
driver:
cores: 0.1
coreLimit: 200m
memory: 512m
labels:
version: 2.4.0
volumeMounts:
- name: test-volume
mountPath: /tmp
executor:
cores: 1
instances: 1
memory: "512m"
labels:
version: 2.4.0
volumeMounts:
- name: test-volume
mountPath: /tmp

28. Example: Kubeflow
“The Kubeflow project is dedicated to making deployments of machine
learning (ML) workflows on Kubernetes simple, portable and scalable.
Our goal is not to recreate other services, but to provide a straightforward
way to deploy best-of-breed open-source systems for ML to diverse infrastructures.
Anywhere you are running Kubernetes, you should be able to run Kubeflow.”
https://www.kubeflow.org/

29. Example: Kubeflow
“The Kubeflow project is dedicated to making deployments of machine
learning (ML) workflows on Kubernetes simple, portable and scalable.
Our goal is not to recreate other services, but to provide a straightforward
way to deploy best-of-breed open-source systems for ML to diverse infrastructures.
Anywhere you are running Kubernetes, you should be able to run Kubeflow.”
Comprehensive Machine Learning Suite.
https://www.kubeflow.org/

30. Kubeflow Features & Integrations
● Chainer Training
● Hyperparameter Tuning (Katib)
● Istio Integration (for TF Serving)
● Jupyter Notebooks
● ModelDB
● ksonnet
● MPI Training
● MXNet Training
● Pipelines
● PyTorch Training
● Seldon Serving
● NVIDIA TensorRT Inference Server
● TensorFlow Serving
● TensorFlow Batch Predict
● TensorFlow Training (TFJob)
● PyTorch Serving

31. Kubeflow Features & Integrations
● Chainer Training
● Hyperparameter Tuning (Katib)
● Istio Integration (for TF Serving)
● Jupyter Notebooks
● ModelDB
● ksonnet
● MPI Training
● MXNet Training
● Pipelines
● PyTorch Training
● Seldon Serving
● NVIDIA TensorRT Inference Server
● TensorFlow Serving
● TensorFlow Batch Predict
● TensorFlow Training (TFJob)
● PyTorch Serving

32. Others
● Aerospike
● Airflow
● ArangoDB
● Cassandra
● CouchDB
● Federation-v2
● Flink
● Gluster
● Kafka
● KubeDB
● MongoDB
● MySQL
● NATS
● PostgreSQL
● Rook
● Velero
● Vitess
● Zookeeper

33. Why Kubernetes?

34. What containers have done for
code, application portability and
reproducible research --
Kubernetes has done for the
orchestration and management
of those things.

35. Complex applications can be
packaged and distributed easily.
If Kubernetes does not provide
the needed primitives, it is easy
enough to extend.

36. Questions?
rkillen@umich.edu
GitHub: @mrbobbytables
Twitter: @mrbobbytables