Learn about the challenges the come with deploying and operating Kubernetes at scale and how the Mesosphere DC/OS Kubernetes integration helps solve them. During this presentation, Joerg Schad discusses: 1. Common challenges associated with getting a Kubernetes cluster up and running 2. The basics of running Kubernetes on Mesosphere DC/OS 3. How failure recovery works with the DC/OS-Kubernetes solution

1. Deploying Kubernetes at Scale
@joerg_schad @dcos

2. Jörg Schad
Technical Community
Lead / Developer
● Core Mesos
developer at
Mesosphere
● Passions are deep
learning, distributed
data systems, and
data analytics

3. Chris Gaun
PMM at Mesosphere /
Kubernetes Expert /
CNCF Ambasador
● Previous to that
Gartner analyst
covering public IaaS
● Kubernetes
community for 3 years

4. Bootcamp: Building Kubernetes-as-a-Service at
Scale, Anywhere
● Episode 1: Building Kubernetes-as-a-Service
at Scale
● Episode 2: Deploying Kubernetes at Scale
with DC/OS
● Episode 3: Kubernetes and Big Data
Services
● Episode 4: Operating Kubernetes at Scale
with DC/OS
● End-to-end components
and best practices
● Automated management
of Kubernetes
● Connecting Kubernetes
to Big Data services
● Delivering an entire
Kubernetes solution

5. © 2018 Mesosphere, Inc. All Rights Reserved.
Agenda
● Introduction Container Orchestration
● Challenges in Kubernetes Deployments
● Kubernetes on DC/OS
● Hands-On
5

6. © 2017 Mesosphere, Inc. All Rights Reserved. 6
In the beginning
there was a big
Monolith

7. © 2016 Mesosphere, Inc. All Rights Reserved. 7

8. © 2018 Mesosphere, Inc. All Rights Reserved.
Hardware
Operating System
Application
8
COMPUTERS

9. © 2018 Mesosphere, Inc. All Rights Reserved.
noun | ˈmīkrō/ /ˈsərvəs/ :
an approach to application development in which a
large application is built as a suite of modular services.
Each module supports a specific business goal and uses
a simple, well-defined interface to communicate with
other modules.*
Microservices are designed to be flexible, resilient,
efficient, robust, and individually scalable.
*From whatis.com
OVERVIEW

10. © 2018 Mesosphere, Inc. All Rights Reserved.
Operating System Operating System Operating System
ServiceApp ServiceServiceAppApp
10
MICROSERVICE
S
- Polyglot
- Single Responsibility
- Smaller Teams
- Utilization
- Machine
types/groups
- Dependency hell
Machine
Infrastructure
Machine Machine
ServiceService ServiceServiceServiceService

11. © 2018 Mesosphere, Inc. All Rights Reserved.
ServiceApp ServiceServiceAppApp
OS
11
CONTAINERS
- Rapid deployment
- Dependency
vendoring
- Container image
repositories
- Spreadsheet
scheduling
OS OS
Machine
Infrastructure
Machine Machine
Container Runtime Container Runtime Container Runtime
ServiceService ServiceServiceServiceService

12. © 2018 Mesosphere, Inc. All Rights Reserved. 12
CONTAINER
SCHEDULING
RESOURCE
MANAGEMENT
SERVICE
MANAGEMENT
- Load Balancing
- Readiness Checking
CONTAINER ORCHESTRATION

13. © 2018 Mesosphere, Inc. All Rights Reserved. 13
CONTAINER
SCHEDULING
- Placement
- Replication/Scaling
- Resurrection
- Rescheduling
- Rolling Deployment
- Upgrades
- Downgrades
- Collocation
RESOURCE
MANAGEMENT
- Memory
- CPU
- GPU
- Volumes
- Ports
- IPs
- Images/Artifacts
SERVICE
MANAGEMENT
- Labels
- Groups/Namespaces
- Dependencies
- Load Balancing
- Readiness Checking
CONTAINER ORCHESTRATION

14. © 2018 Mesosphere, Inc. All Rights Reserved.
Orchestration
14
Machine Infrastructure
Web Apps & Services
Scheduling
Resource Management
Container Runtime
Machine & OS
Service Management
CONTAINER
ORCHESTRATIO
N
Machine & OS Machine & OS
Container Runtime Container Runtime

15. © 2017 Mesosphere, Inc. All Rights Reserved. 15
Challenges

16. - RBAC + IAM
- Network segmentation
- OSS framework/ container control
- Multiple isolated clusters
- Vanilla / standard / OSS Kubernetes
- Identical setup / components / no snowflakes
- Multi/hybrid cloud
- Multi data center
- Easy updating / versioning
- Scaling
- Integrations - ELB, networking, Storage, Monitoring
- Easy installation
Security
Interoperability
Easy
Management
Delivering Kubernetes Challenges
Where they run Kubernetes
Multiple management hurdles
42%
On
Prem
57%
DIY
AWS
~40%
40+%
Require more security for wider
deployment
CNCF poll

17. Poll Question
1. What phase is your organization’s Kubernetes
journey?
a. Have not started
b. Actively researching container orchestration
options
c. Planning a production Kubernetes project
d. Managing a production Kubernetes cluster

18. © 2018 Mesosphere, Inc. All Rights Reserved. 18
Kubernetes
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime Container RuntimeContainer Runtime
Machine & OS Machine & OS
Container Runtime
Orchestration
Scheduling
Resource Management
Service Management
Web Apps & Services
Machine Infrastructure
OrchestrationScheduling
Resource Management
Service Management
Web Apps & Services

19. © 2018 Mesosphere, Inc. All Rights Reserved.
KUBERNETES ARCHITECTURE
19

20. © 2018 Mesosphere, Inc. All Rights Reserved. 20
Kubernetes HA
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime Container RuntimeContainer Runtime
Machine & OS Machine & OS
Container Runtime
Machine Infrastructure
Web Apps & Services
etcd etcdetcd
api-
server
api-
server
api-
server
Kube-
proxy
Kube-
proxy
scheduler
Kube-
proxy
scheduler
schedulercontrollercontroller
controller
kubelet kubelet kubelet
...

21. © 2018 Mesosphere, Inc. All Rights Reserved. 21
Kubernetes HA
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime Container RuntimeContainer Runtime
Machine & OS Machine & OS
Container Runtime
Machine Infrastructure
Web Apps & Services
etcd etcdetcd
api-
server
api-
server
api-
server
Kube-
proxy
Kube-
proxy
scheduler
Kube-
proxy
scheduler
schedulercontrollercontroller
controller
kubelet kubelet kubelet
...

22. © 2017 Mesosphere, Inc. All Rights Reserved. 22
Datacenter

23. © 2017 Mesosphere, Inc. All Rights Reserved. 23
Datacenter
Typical Datacenter
siloed, over-provisioned servers,
low utilization
Kubernetes
Jenkins
Kafka
Spark
CockroachDB

24. © 2016 Mesosphere, Inc. All Rights Reserved. 24

25. © 2017 Mesosphere, Inc. All Rights Reserved. 25
Datacenter
Typical Datacenter
siloed, over-provisioned servers,
low utilization
Mesos/ DC/OS
automated schedulers, workload multiplexing onto the
same machines
Kubernetes
Jenkins
Kafka
Spark
Marathon

26. © 2016 Mesosphere, Inc. All Rights Reserved. 26
DC/OS
PHYSICAL
INFRASTRUCTURE
MICROSERVICES, CONTAINERS, & DEV
TOOLS
VIRTUAL MACHINES PUBLIC CLOUDS
DATA SERVICES, MACHINE LEARNING, & AI
Security &
Compliance
Application-Aware
Automation
Multitenancy
Hybrid Cloud
Management
100+
MOR
E
DatacenterEdge
Datacenter and Cloud as a Single Computing Resource
Powered by Apache Mesos
20+
MOR
E

27. © 2016 Mesosphere, Inc. All Rights Reserved.
Two-level Scheduling
1. Agents advertise resources to Master
2. Master offers resources to Framework
3. Framework rejects / uses resources
4. Agent reports task status to Master
27
MESOS ARCHITECTURE
Mesos
Master
Mesos
Master
Mesos
Master
Mesos AgentMesos Agent Service
Cassandra
Executor
Cassandra
Task
Kubernetes
Scheduler
Spark
Executor
Spark
Task
Mesos AgentMesos Agent Service
Docker
Executor
Docker
Task
K8s Executor
Kubelet
Task
Marathon
Scheduler
Kafka
Scheduler

28. © 2018 Mesosphere, Inc. All Rights Reserved. 28
DC/OS and Kubernetes
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime Container RuntimeContainer Runtime
Machine & OS Machine & OS
Container Runtime
Orchestration
Scheduling
Resource Management
Service Management
Web Apps & Services
Machine Infrastructure
Mesosphere DC/OS
OrchestrationScheduling
Resource Management
Service Management
Web Apps & Services

29. © 2018 Mesosphere, Inc. All Rights Reserved. 29
DC/OS and Kubernetes
Orchestration
Machine Infrastructure
Web Apps & Services
Scheduling
Resource Management
Container Runtime
Machine & OS
Service Management
Machine & OS Machine & OS
Container Runtime Container RuntimeContainer Runtime
Machine & OS Machine & OS
Container Runtime
Orchestration
Scheduling
Resource Management
Service Management
Web Apps & Services
Machine Infrastructure
Mesosphere DC/OS
Big Data
Services
● Spark
● Flink
● Kafka
● ….

30. © 2018 Mesosphere, Inc. All Rights Reserved. 30
Mesos and Kubernetes
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime Container RuntimeContainer Runtime
Machine & OS Machine & OS
Container Runtime
Machine Infrastructure
DC/OS / Apache Mesos
Web Apps & Services
etcd etcdetcd
api-
server
api-
server
api-
server
Kube-
proxy
Kube-
proxy
scheduler
Kube-
proxy
scheduler
schedulercontrollercontroller
controller
kubelet kubelet kubelet
...

31. © 2018 Mesosphere, Inc. All Rights Reserved. 31
Mesos and Kubernetes
Machine Infrastructure
Container Runtime
Machine & OS Machine & OS Machine & OS
Container Runtime Container RuntimeContainer Runtime
Machine & OS Machine & OS
Container Runtime
Machine Infrastructure
Mesosphere Apache Mesos
Web Apps & Services
etcd etcdetcd
api-
server
api-
server
api-
server
Kube-
proxy
Kube-
proxy
scheduler
Kube-
proxy
scheduler
schedulercontrollercontroller
controller
kubelet kubelet kubelet
...

32. © 2018 Mesosphere, Inc. All Rights Reserved. 32
Mesos and Kubernetes

33. © 2017 Mesosphere, Inc. All Rights Reserved. 33
DC/OS
Kubernetes
Integration

34. © 2018 Mesosphere, Inc. All Rights Reserved.
● Unaltered upstream Kubernetes, unlike OpenShift
● Stand-alone Kubernetes clusters are exposed, not a PaaS using Kubernetes
● Simple install onto a DC/OS cluster
● Integration with DC/OS data services
● HA and Secure by default in EE
● Customers have choice between Marathon and Kubernetes
DC/OS Kubernetes
34

35. © 2018 Mesosphere, Inc. All Rights Reserved. 35
Networking in Kubernetes on DC/OS

36. © 2018 Mesosphere, Inc. All Rights Reserved. 36

37. © 2018 Mesosphere, Inc. All Rights Reserved. 37

38. © 2018 Mesosphere, Inc. All Rights Reserved. 38

39. © 2018 Mesosphere, Inc. All Rights Reserved. 39

40. © 2018 Mesosphere, Inc. All Rights Reserved. 40

41. © 2018 Mesosphere, Inc. All Rights Reserved. 41

42. © 2018 Mesosphere, Inc. All Rights Reserved. 42

43. © 2018 Mesosphere, Inc. All Rights Reserved. 43

44. Download Now
https://mesosphere.com/resource/category/ebook/

45. © 2017 Mesosphere, Inc. All Rights Reserved. 45
THANK YOU!
ANY
QUESTIONS?
@dcos
users@dcos.io
/groups/8295652
/dcos
/dcos/examples
/dcos/demos
chat.dcos.io
https://github.com/mesosphere/dcos-kubernetes-quickstart
https://mesosphere.com/blog/another-kubernetes-service/