Kubernetes Introdution
Production-Grade Container Orchestration
Jan 2017, Peng Xiao
Network Consulting Engineer
Cisco Systems
8

https://research.google.com/pubs/pub35290.html

In The Very Beginning…
Hardware
Application
Operating System

Scale & High Availability
Hardware
App
Operating System
Hardware
App
Operating System
Hardware
App
Operating System

Hardware Virtualization
Infratructure
App
Operating System
App
Operating System
App
Operating System
Machine Machine Machine

Containerized
Infratructure
App
Operating System
App
Operating System
App
Operating System
Machine Machine Machine
Container Runtime Container Runtime Container Runtime

Container Orchestration
Infratructure
App
Operating System Operating System Operating System
Machine Machine Machine
Container Runtime Container Runtime Container Runtime
Container Orchestration
App App App App

Container Orchestration
Infratructure
Operating System Operating System Operating System
Machine Machine Machine
Container Runtime Container Runtime Container Runtime
Resource Management
Scheduling
Service Management
Apps
Orchestration

Container Orchestration
• Schedule containers to physical/virtual machines
• Restart containers if they stop
• Provide private container network
• Scale up and down
• Service discovery

Container Orchestration War?

Kubernetes
• Greek for “Helmsman”; also the root of
the word “governor” and “cybernetic”
• Orchestrator for containers
• Builds on Docker containers
• Also supporting other container technologies
• Multi-cloud and bare-metal environments
• Inspired and informed by Google’s
experiences and internal systems
• 100% Open Source, written in Go.
• Release 1.0 21th July 2015
Large-scale cluster management at Google with Borg https://research.google.com/pubs/pub43438.html

Velocity
1.0
1.1
1.2
1.3
TotalCommits
1.5
Commits Since July 2014
1.4

Kubernetes Architecture

Quick Recap
• Docker
• Docker Compose
• Docker Swarm

Kubernetes Architecture

Setup Kubernetes Environment
• Minikube
• Simplest way to get Kubernetes cluster up and running
• Support Microsoft Windows and Mac OSX
• Kubernetes Multi-Node Cluster
• Emulates production environment
• Good for testing advanced scenarios
• Google Container Engine
• Hosted and managed by Google
• Powered by Google Compute Engine

Getting Started with Minikube
• Install Oracle VirtualBox for Mac
• Install Docker Toolbox for Mac
• Install Docker Version Manager
• Install the latest version of Minikube for Mac OSX
• Download the latest version of kubectl from this link
• Run the following commands from the directory where kubectl is downloaded
• chmod +x ./kubectl
• sudo mv kubectl /usr/local/bin
• Launch minikube with the following command:
• minikube start –wm-driver=virtualbox
• Test minikube installation with the following commands
• minikube status
• kubectl get cs

kubeadm
master.myco.com# apt-get install -y kubelet kubeadm kubectl kubernetes-cni
master.myco.com# kubeadm init
Kubernetes master initialized successfully!
You can now join any number of nodes by running the following command:
kubeadm join --token 48b69e.b61e2d0dd5c 10.140.0.3
node-01.myco.com# apt-get install -y kubelet kubeadm kubectl kubernetes-cni
node-01.myco.com# kubeadm join --token 48b69e.b61e2d0dd5c 10.140.0.3
Node join complete.
master.myco.com# kubectl apply -f https://git.io/weave-kube
Network setup complete.

Kubernetes Master

Kubernetes Node

Kubernetes Pod
• Group of one or more containers that are always co-located, co-
scheduled, and run in a shared context
• Containers in the same pod have the same hostname
• Each pod is isolated by
• ○ Process ID (PID) namespace
• ○ Network namespace
• ○ Interprocess Communication (IPC) namespace
• ○ Unix Time Sharing (UTS) namespace
• Alternative to a VM with multiple processes

Kubernetes Pod
• Containers within the same pod communicate with each other using
IPC
• Containers can find each other via localhost
• Each container inherits the name of the pod
• Each pod has an IP address in a flat shared networking space
• Volumes are shared by containers in a pod

Deploying a pod

Services
• An abstraction to define a logical set of Pods bound by a policy by to
access them
• Services are exposed through internal and external endpoints
• Services can also point to non-Kubernetes endpoints through a
Virtual-IP-Bridge
• Supports TCP and UDP
• Interfaces with kube-proxy to manipulate iptables
• Service can be exposed internal or external to the cluster

Service Types
• ClusterIP
• Service is reachable only from inside of the cluster
• NodePort
• Service is reachable through <NodeIP>:NodePort address.
• LoadBalancer
• Service is reachable through an external load balancer mapped to
<NodeIP>:NodePort address

Service Discovery -Environment variables
• Kubernetes creates Docker Link compatible environment variables in
all Pods
• Containers can use the environment variable to talk to the service
endpoint

Service Discovery - DNS
• The DNS server watches Kubernetes API for new Services
• The DNS server creates a set of DNS records for each Service
• Services can be resolved by the name within the same namespace
• Pods in other namespaces can access the Service by adding the
namespace to the DNS path
• my-service.my-namespace

Replication Controller
• Ensures that a Pod or homogeneous set of Pods are always up and
available
• Always maintains desired number of Pods
• If there are excess Pods, they get killed
• New pods are launched when they fail, get deleted, or terminated
• Creating a replication controller with a count of 1 ensures that a Pod
is always available
• Replication Controller and Pods are associated through Labels

Scaling Pods with Replication Controller

Replica Set
• Replica Sets are the next generation Replication Controllers
• Ensures specified number of pods are always running
• Pods are replaced by Replica Sets when a failure occurs
• New pods are automatically scheduled
• Labels and Selectors are used for associating Pods with Replica Sets
• Usually combined with Pods when defining the deployment

Kubernetes Networking

Docker Networking

Kubernetes Networking
• Highly-coupled container-to-container communications
• Pod-to-Pod communications
• Pod-to-Service communications
• External-to-internal communications

Container to Container
• All containers within a pod can reach each other’s port on localhost

Pod to Pod
• Kubernetes imposes the following fundamental
requirements on any networking implementation
• all pods (containers) can communicate with all other
containers without NAT
• all nodes can communicate with all containers (and vice-
versa) without NAT
• the IP that a container sees itself as is the same IP that
others see it as
• Network model
• Can be L3 routed
• Can be underlayed (cloud)
• Can be overlayed (SDN)

Pod to Pod: How?
• On GCE/GKE
• GCE Advanced Routes (program the fabric)
• “Everything to 10.1.1.0/24, send to this VM”
• Plenty of other ways
• AWS: Route Tables
• Weave
• Calico
• Flannel
• OVS
• OpenContrail
• Cisco Contiv
• Others...

Pod to Service

Adoption
~4k Commits
in 1.5
+25% Unique
Contributors
Top 0.01% of all
Github Projects
3500+ External
Projects Based
on K8s
Companies
Contributing
Companies
Using

Thanks for your time