Mais conteúdo relacionado Semelhante a PKS: The What and How of Enterprise-Grade Kubernetes (20) Mais de VMware Tanzu (20) PKS: The What and How of Enterprise-Grade Kubernetes1. PKS: The What and How of Enterprise-Grade
Kubernetes
(or: Because the Cool Kids All Spell Kontainer with a K ;-))
Cornelia Davis, Sr. Director of Technology, Pivotal, @cdavisafc
Fred Melo, Director of Technology, Pivotal, @fredmelo_br
1
3. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Safe Harbor Statement
The following is intended to outline the general direction of Pivotal's offerings. It is
intended for information purposes only and may not be incorporated into any
contract. Any information regarding pre-release of Pivotal offerings, future updates
or other planned modifications is subject to ongoing evaluation by Pivotal and is
subject to change. This information is provided without warranty or any kind, express
or implied, and is not a commitment to deliver any material, code, or functionality,
and should not be relied upon in making purchasing decisions regarding Pivotal's
offerings. These purchasing decisions should only be based on features currently
available. The development, release, and timing of any features or functionality
described for Pivotal's offerings in this presentation remain at the sole discretion of
Pivotal. Pivotal has no obligation to update forward looking information in this
presentation.
3
5. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Companies have Many Types of Workloads
5
CONTAINERS
EVENT-DRIVEN
FUNCTIONS
DATA SERVICES
MICROSERVICES
Batches
MONOLITHIC
APPLICATIONS
6. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
…And Different SDLCs
6
“Stable”
maintenance
Active
Development
RUN BUILD + RUN
7. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 7
Monolithic
PAAS
Cloud Native
RUN BUILD + RUN
8. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 8
Traditional IaaS
Cloud-native App
Platform
Monolithic
PAAS
Cloud Native
RUN BUILD + RUN
9. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 9
Infrastructure
Orchestrated CaaS
Cloud-native App
Platform
Monolithic
PAAS
Cloud Native
RUN BUILD + RUN
Traditional IaaS
10. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PCF2.0
CONFIDENTIAL 10
Infrastructure
Orchestrated CaaS
Cloud-native App
Platform
Monolithic
PAAS
Cloud Native
RUN BUILD + RUN
Traditional IaaS
11. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CONFIDENTIAL 11
Infrastructure
Traditional
IaaS Orchestrated CaaS
Cloud-native App
Platform
Monolithic
PAAS
Cloud Native
PCF2.0
RUN BUILD + RUN
12. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
12
Hardware
IaaS
Container Orchestrator
Application
Platform
Serverless
Functions
Strategic goal: Push as many workloads as technically
feasible to the top of the platform hierarchy
Higher flexibility and
less enforcement of
standards
Lower development
complexity and higher
operational efficiency
14. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 14
15. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Operational Challenges with any platform
15
Patches Patching platform components with
thousands of apps running should feel normal.
Scaling Seamlessly scale platform components
to accommodate changing demand.
Upgrades. How do you roll out new versions of
the platform with the lights on?
Operating Effort Operating a platform should
require very few resources and minimum manual
intervention. Otherwise, is it really providing
operational benefits?
Multi-cloud Provide a reliable and smooth
experience for any cloud.
Open APIs Allow platform operations from
different toolsets and the creation of CD
pipelines.
Consistency Provide a consistent setup
experience, across different cloud environment
configurations.
Setup time How long does it take to setup a real
world working environment? Think hours, not
weeks.
Day 1 - Build Day 2 - Operate
16. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Kubernetes - especially hard to operationalize
16
High Availability. No out-of-the-box fault-
tolerance for the cluster components themselves
(masters and etcd nodes).
Scaling. Kubernetes clusters handle scaling the
pod/service within the Nodes, but doesn’t provide
a mechanism to scale Masters & etcd VMs.
Health checks and healing. The Kubernetes
cluster does routine health checks for the health
of Nodes only.
Upgrades. Rolling upgrades on a large fleet of
clusters is hard. Who manages the system it runs
on?
17. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
PIVOTAL CLOUD FOUNDRY OPS
Powered by BOSH
17
BOSH is an open source tool
for release engineering,
deployment, lifecycle
management, and monitoring
of distributed systems.
Packaging w/ embedded OS
Server provisioning on any IaaS
Software deployment across availability
zones
Health monitoring (server AND processes)
Self-healing w/ Resurrector
Storage management
Rolling upgrades via canaries
Easy scaling of clusters
19. Project Kubo
Uniform way to instantiate,
deploy, and manage highly
available Kubernetes
clusters. On any cloud.
Launched by Pivotal & Google
Feb 2017, Donated to Cloud
Foundry Foundation June 2017
Committers: Pivotal, Google,
VMware
“Day 1” Build
" Deploy Kubernetes cluster via
BOSH
“Day 2” Operate
" Self-healing VMs and monitoring via
BOSH
" Elastic scaling for clusters
" Rolling upgrades to latest
Kubernetes release
" High-availability and multi-AZ
support
21. Pivotal Container Service (PKS)
Provides the control plane
for provisioning and
managing Kubo releases
Joint development effort
between Pivotal and
VMWare
Kubernetes Dial Tone:
• Health management
• Framework for Metrics and Logging
• Autoscaling
• Persistence interface
• Networking interface
Control Plane:
• Provisioning Engine
• Self-service Clusters
• Software Update Automation
• Load balancing
• Networking
• Persistence
• Multi-tenancy
24. 24
PKS: Networking (different options available)
BOSH network
Worker Worker
10.0.30.12
ContainerContainerContainer
10.200.2.6C2C overlay 10.200.1.510.200.1.4
10.0.30.11
kube-proxy
iptables
kube-proxy
iptables
Service network
Service
25. BOSH
GCP
Service
Broker
Harbor
NSX-T
Kubernetes
K8s Cluster
K8s Cluster
K8s Cluster
Built with open-source Kubernetes — Constant compatibility
with the current stable release of Kubernetes, operated by
BOSH. No proprietary extensions.
Production-ready — Highly available from apps to
infrastructure, no single points of failure. Built-in health checks,
scaling, auto-healing and rolling upgrades.
Multicloud — BOSH provides a reliable and consistent
operational experience. For any cloud.
Network management and security out-of-the-box with
VMware NSX-T. Multi-cloud, multi-hypervisor.
GCP APIs access — The GCP Service Broker allows apps to
transparently access Google Cloud APIs, from anywhere. Easily
move workloads to/from Google Container Engine (GKE).
Fully automated Ops — Fully automated deploy, scale, patch,
upgrade. No downtime. Use CD pipelines to deploy your
platform, too.
VMware GCP Azure Openstack AWS
PKSController
26. Apps are constantly compatible between PKS and Google Container Engine (GKE)
Google Container
Engine (GKE)
Pivotal Container
Service (PKS)
Private and
Public Cloud
Public Cloud
(on GCP)
28. Leveraging more than one abstraction
Other
Broker
Services
Platform Services
Logging Metrics Monitoring
Pivotal
Application Service
Application Application
VMware GCP Azure Openstack AWS
PKSController
GCP
Service
Broker
Harbor
NSX-T
Kubernetes
K8s Cluster
K8s Cluster
K8s Cluster
31. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 31
BOSH
GCP
Service
Broker
Harbor
NSX-T
Kubernetes
K8s Cluster
K8s Cluster
K8s Cluster
Built with open-source Kubernetes — Constant compatibility
with the current stable release of Kubernetes, operated by
BOSH. No proprietary extensions.
Production-ready — Highly available from apps to
infrastructure, no single points of failure. Built-in health checks,
scaling, auto-healing and rolling upgrades.
Multicloud — BOSH provides a reliable and consistent
operational experience. For any cloud.
Network management and security out-of-the-box with
VMware NSX-T. Multi-cloud, multi-hypervisor.
GCP APIs access — The GCP Service Broker allows apps to
transparently access Google Cloud APIs, from anywhere. Easily
move workloads to/from Google Container Engine (GKE).
Fully automated Ops — Fully automated deploy, scale, patch,
upgrade. No downtime. Use CD pipelines to deploy your
platform, too.
VMware GCP Azure Openstack AWS
PKSController How does this all work?
• Availability and Resiliency
• Operational Efficiency
• Multi-tenancy isolation
• Security
• Extensibility
Guiding Principles
32. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
32
BOSH
PKSControlPlane
Pivotal Container Service
33. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
BOSH
Pivotal Container Service
PKSControlPlane
Create cluster
34. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 34
Availability Zone B
Availability Zone A
HA and Health Management
- Kubelet watches and restart containers
- Bosh director watches and restarts nodes
- Bosh agent watches and restarts processes
- Bosh distributes deployments across AZ’s
Kubelet
Kube-proxy
Pod
Pod
K8s Node
Pod
API Server
Kube Scheduler
K8s Master
Controller
Manager
Bosh agent
Bosh agentBosh director
Watches and restarts VMs
Availability Zone A
Availability Zone B
Create cluster
35. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
35
BOSH
Pivotal Container ServicePKSControlPlane
Kubelet
Kube-proxy
Fluentd
Pod
Pod
Pod
K8s Node
Kubelet
Kube-proxy
Fluentd
Pod
Pod
Pod
K8s Node
Kubelet
Kube-proxy
Pod
Pod
Pod
K8s Worker
API Server Kube Scheduler
K8s Master
Controller Manager
Node
Controller
Replication
Controller
Endpoints
Controller
Service Accounts
& Token
Etcd
GCP
Service
BrokerHarbor
NSX-T
Proxy
NCP
T1 Routers
NSX-T
T1 Routers
Other
components
Included component, but optional usage
CFCR / Kubo
36. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
36
BOSH
Pivotal Container ServicePKSControlPlane
Kubelet
Kube-proxy
Fluentd
Pod
Pod
Pod
K8s Node
Kubelet
Kube-proxy
Fluentd
Pod
Pod
Pod
K8s Node
Kubelet
Kube-proxy
Pod
Pod
Pod
K8s Worker
API Server Kube Scheduler
K8s Master
Controller Manager
Node
Controller
Replication
Controller
Endpoints
Controller
Service Accounts
& Token
Etcd
GCP
Service
BrokerHarbor
NSX-T
Proxy
NCP
T1 Routers
NSX-T
T1 Routers
Other
components
Included component, but optional usage
CFCR / Kubo
Kubernetes
37. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
Kubernetes components
API Server
Kube Scheduler
K8s Master
Controller
Manager
Etcd
K-V Store
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
CNI
Persistent Volume
Virtual switch
Network edge router
Volume mount
Aws, Azure, GCP, vSphere
SCSI, FC, NFS, Photon, Scale IO, …
Provisioner
NSX-T, Flannel, Calico, GCE,
Nuange, OVN, Kube-router…
CNI Plugin
Protobuf
Network edge routerNetwork edge router
38. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
38
API Server
Kube Scheduler
K8s Master
Controller
Manager
Etcd
K-V Store
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
CNI
Persistent Volume
Virtual switch
Network edge router
Volume mount
Aws, Azure, GCP, vSphere
SCSI, FC, NFS, Photon, Scale IO, …
Provisioner
NSX-T, Flannel, Calico, GCE,
Nuage, OVN, Kube-router…
CNI Plugin
Protobuf
Network edge routerNetwork edge router
Pluggable Networking (CNI)
39. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 39
BOSH
Pivotal Container Service
PKSController
GCP
Service
Broker
BOSH network
10.0.1.5
Harbor
10.0.1.2
Etcd
10.0.1.6
Etcd
10.0.1.6
API Server Kube Scheduler
K8s Master
Controller Manager
Node
Controller
Replication
Controller
Endpoints
Controller
Service Accounts
& Token
10.0.1.4
Kubelet
Kube-proxy
Fluentd
Pod
Pod
Pod
K8s Node
10.0.1.3
Kubelet
Kube-proxy
Fluentd
Pod
Pod
Pod
K8s Node
10.0.1.3
Pod
Pod
Pod
K8s Worker
Kubelet
Kube-proxy
10.0.1.3
Networking
40. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
40
BOSH
PKSController
GCP
Service
Broker
Harbor
Etcd
API Server Kube Scheduler
K8s Master
Controller Manager
Node
Controller
Replication
Controller
Endpoints
Controller
Service Accounts
& Token
Base Network
10.0.0.11
10.0.0.16
10.0.0.17
10.0.0.13
BOSH Network
NSX-T
Mgr
NSX-T
NCP
.18
.19
10.0.0.21
10.0.0.22
Kubelet
NSX
Kube-Proxy
OVS
K8s Worker
K8s Worker
Kubelet
41. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
41
BOSH
PKSController
GCP
Service
Broker
Harbor
Etcd
API Server Kube Scheduler
K8s Master
Controller Manager
Node
Controller
Replication
Controller
Endpoints
Controller
Service Accounts
& Token
Base Network
10.0.0.11
10.0.0.16
10.0.0.17
10.0.0.13
BOSH Network
NSX-T
Mgr
NSX-T
NCP
.18
.19
10.0.0.21
10.0.0.22
Kubelet
Pod
Pod
NSX
Kube-Proxy
OVS
K8s Worker
Pod
Pod
NSX
Kube-Proxy
K8s Worker
K8K8s NameSpace A = NSX Logical Switch
K8K8s NameSpace B = NSX Logical Switch
T1 10.172.10.2
10.172.100.0/24
192.168.100.0/24
NO_NAT
NAT
T1 10.172.10.3
T0 10.172.10.1
Kubelet
OVS
NSX CNI PLUGIN
Overlay Networks
42. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 42
-T
Pivotal Container Service
Network Policy Management
Multi-Tenancy Isolation
Network Security Groups
Dynamic Load Balancing &
Ingress
Comprehensive VM +
Container security model
43. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 43
kubectl apply -f <file.yml>
Network Policies Management
BOSH
Pivotal Container Service
PKSController
NSX-T
Proxy
NCP
T1 RoutersT1
Routers
Other
components
NSX-T
Kubelet
Kube-
proxy
Fluentd
Pod
Pod
Pod
K8s Node
Kubelet
Kube-
proxy
Fluentd
Pod
Pod
Pod
K8s NodeNode
NSX Router
Network policy management
API Server
Kube Scheduler
K8s Master
Controller
Manager
Etcd
new policy
Apply
Kubelet
K. Proxy
Pod
Pod
Pod
44. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
44
Deployment Topologies & Multi-Tenancy
Multi-cluster Single cluster
K8s Cluster A
K8s Cluster
BOSH
PKSController
Namespace A
Namespace B
Namespace C
BOSH
PKSController
K8s Cluster B
K8s Cluster C
45. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
45
admin@k8s-master:~$ kubectl create namespace foo
namespace ”foo" created
admin@k8s-master:~$ kubectl create namespace bar
namespace ”bar" created
admin@k8s-master:~$ kubectl run nginx-foo --image=nginx -n foo
deployment "nginx-foo" created
admin@k8s-master:~$ kubectl run nginx-bar --image=nginx -n bar
deployment "nginx-bar" created
Namespace: foo Namespace: bar
NSX / K8s topology
10.24.0.0/24 10.24.1.0/24 10.24.2.0/24
NAT boundary NAT boundary
K8s nodesK8s Masters
NameSpace as a Tenancy Construct
NSX-T Automatically Provides Each Namespace a secure network & routing
46. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
46
admin@k8s-master:~$ kubectl label pods nginx-foo-3492604561-nltrf secgroup=web -n foo
Pod "nginx-nsx-3492604561-nltrf" labeled
admin@k8s-master:~$ kubectl label pods nginx-bar-2789337611-z09x2 secgroup=db -n bar
pod "nginx-k8s-2789337611-z09x2" labeled
admin@k8s-master:~$ kubectl get pods --all-namespaces -Lsecgroup
NAMESPACE NAME READY STATUS RESTARTS AGE SECGROUP
k8s nginx-foo-2789337611-z09x2 1/1 Running 0 58m web
nsx nginx-bar-3492604561-nltrf 1/1 Running 0 1h db
Namespace: foo Namespace: bar
NSX / K8s topology
10.24.0.0/24 10.24.1.0/24 114.4.10.0/26
NAT boundary NAT boundary
Web
• Security Groups are defined in NSX with ingress and
egress policy
• Each Security Group could be micro-segmented to
protect Pods from each other
DB
47. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 47
App / Container Deployment, Services & Routing
cf push
App
myapp.mydomain.net
Pivotal Cloud Foundry
Application Runtime
Pivotal Container
Service
kubectl run
????
????
48. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 48
App / Container Deployment, Services & Routing
cf push
App
myapp.mydomain.net
Pivotal Cloud Foundry
Application Runtime
Pivotal Container
Service
kubectl run
????
????
49. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 49
BOSH
Pivotal Container Service
PKSController
Kubelet
Kube-proxy
Pod
K8s Worker
K8s Master
Controller Manager
Node
Controller
Replication
Controller
Endpoints
Controller
Service Accounts &
Token
kubectl run
<image>
Harbor
Etcd
Pod
API Server Kube Scheduler
Pod
Pod
50. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 50
???
kubectl run
<docker img>
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
Pod
192.168.0.4
192.168.0.3
192.168.0.2
API Server Kube Scheduler
K8s Master
Controller Manager
API Server Kube Scheduler
K8s Master
Controller Manager
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
Pod
192.168.0.4
192.168.0.3
192.168.0.2
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
192.168.0.4
192.168.0.3
Pod
192.168.0.2
K8s Cluster
Base Network
Overlay Network
Cluster network
(conceptual)
51. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 51
kubectl run
<docker img>
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
Pod
192.168.0.4
192.168.0.3
192.168.0.2
Cluster network
(conceptual)
API Server Kube Scheduler
K8s Master
Controller Manager
API Server Kube Scheduler
K8s Master
Controller Manager
K8s Cluster
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
Pod
192.168.0.4
192.168.0.3
192.168.0.2
Base Network
Overlay Network
Service
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
192.168.0.4
192.168.0.3
Pod
192.168.0.2
52. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 52
NodePort
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
Pod
192.168.0.4
192.168.0.3
192.168.0.2
Cluster network
(conceptual)
API Server Kube Scheduler
K8s Master
Controller Manager
API Server Kube Scheduler
K8s Master
Controller Manager
K8s Cluster
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
Pod
192.168.0.4
192.168.0.3
192.168.0.2
Base Network
Overlay Network
kubectl expose
deployment
NodePort
NodePort
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
192.168.0.4
192.168.0.3
Pod
192.168.0.2
host:port
NodePort
Load Balancer
host
53. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 53
NodePort
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
Pod
192.168.0.4
192.168.0.3
192.168.0.2
API Server Kube Scheduler
K8s Master
Controller Manager
API Server Kube Scheduler
K8s Master
Controller Manager
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
Pod
192.168.0.4
192.168.0.3
192.168.0.2
kubectl expose
deployment
Load Balancer
NodePort
K8s Worker
Kubelet
Kube-proxy
Pod
Pod
192.168.0.4
192.168.0.3
Pod
192.168.0.2
NodePort
Cloud Load
Balancer
host
Dynamically
provisioned
Kubernetes
Cluster
Cloud
35.190.151.218
Public IP
-T
54. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
54
Services: Ingress
An API object that manages external access to the services in a cluster, typically HTTP.
Ingress can provide load balancing, SSL termination and name-based virtual hosting.
K8s Worker
Kubelet
Kube-proxy
PodPodPod
K8s Worker
Kubelet
Kube-proxy
PodPodPodS2
S1
K8s Worker
Kubelet
Kube-proxy
PodPod
Pod
Ingress
/foo
/bar
foo.bar.com
Deploy
55. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
5
BOSH
Pivotal Container
Service
PKSController
Kubelet
Kube-proxy
Fluentd
Pod
Pod
K8s Node
API Server Kube Scheduler
K8s Master
Controller Manager
Node
Controller
Replication
Controller
Endpoints
Controller
Service Accounts &
Token
Pod
192.168.0.1
192.168.0.4
192.168.0.3
192.168.0.2
Pivotal Application
Service Route-sync
GO Router
myapp.apps.myorg.net
kubectl expose deployment
(NodePort w/ route sync)
NodePort
Integration with CFAR Routing
56. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
56
API Server
Kube Scheduler
K8s Master
Controller
Manager
Etcd
K-V Store
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
Kubelet
Kube-proxy
K8s Worker
Pod
Pod
Pod
CNI
Persistent Volume
Virtual switch
Network edge router
Volume mount
Aws, Azure, GCP, vSphere
SCSI, FC, NFS, Photon, Scale IO, …
Provisioner
Protobuf
Network edge routerNetwork edge router
Pluggable Persistent Volumes
57. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 57
Pod
Container Container
Host mount Fiber Channel,
iSCSI, NFS
GCP, Azure,
AWS, vSphere
Volumes and Persistent Volumes
Legacy Systems
Stateful Systems
Database
58. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 58
kubectl apply -f <file.yml>
Pivotal Container Service
PKSController
K8s Master Etcd
Worker
Kubelet
K. Proxy
Pod
Pod
Worker
Kubelet
K. Proxy
Pod
Pod
Worker
Kubelet
K. Proxy
Pod
Pod
BOSH
60. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
60
Stateful Sets
Stable, unique network identifiers
Stable, persistent storage
Ordered, graceful deployment and scaling
Ordered, automated rolling updates
* stable = survives Pod rescheduling
Pod
Stateful set
Name = web
Replicas = 3
Pod
web-0
web-1
web-2
Pod
PV-claim-web-0
Each Pod gets an unique ID and Persistent Volumes
PV-claim-web-1
PV-claim-web-2
61. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons
Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 61
Stateful Sets
Apps / clusters using specific IPs / names
to communicate
Apps / clusters relying on persistent and
durable storage on specific mount points
Apps / clusters with specific starting order
for components
Stable, unique network identifiers
Stable, persistent storage
Ordered, graceful deployment and scaling
Ordered, automated rolling updates
* stable = survives Pod rescheduling
Clustered, stateful workloads or legacy apps