Load balancing is an important part of any resilient web application. Kubernetes supports a few options for external load balancing, but they are limited in features. After a brief discussion of those options and the features they lack, we’ll show how to build an advanced load balancing solution for Kubernetes on top of NGINX, utilizing Kubernetes features including Ingress, Annotations, and ConfigMap. We’ll conclude with a demo of how to use NGINX and NGINX Plus to expose services to the Internet. Sched Link: http://sched.co/6Bc9

1. Creating an Advanced Load Balancing
Solution for Kubernetes with NGINX
Andrew Hutchings — Technical Product Manager, NGINX, Inc., @LinuxJedi

2. About LinuxJedi
• Kubernetes user for 4 days
• Worked at HP on OpenStack LBaaS and ATG
• Worked on several Open Source DBs
• Alopecia sufferer

3. Goals
• Basic and advanced load balancing
• Current load balancing options in Kubernetes
• Ingress resource
• Implementing an Ingress controller for NGINX
• Load balancing demo: exposing Kubernetes services to the Internet

4. Basic Load Balancing
A load balancer
distributes request
among healthy servers
LB
Server 1 Server 2 Server 3

5. Basic Load Balancing
HTTPHTTP
Layer 7
TCPTCP UDPUDP
Layer 4

6. Advanced Load Balancing
• SSL termination
• Active health checks
• Security
• Bandwidth limits
• Logging
• Real-time statistics
• Session Persistence
• Content-based routing
• and more…

7. Load Balancing in Kubernetes
Internal
• kube-proxy
External
• NodePort
• LoadBalancer
• External IPs
• Service loadbalancer
• Ingress

8. Internal: Kube-proxy
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: backend
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: backend
# env | grep -i backend
BACKEND_SERVICE_SERVICE_HOST=10.3.246.245
BACKEND_SERVICE_SERVICE_PORT=80
…
# env | grep -i backend
BACKEND_SERVICE_SERVICE_HOST=10.3.246.245
BACKEND_SERVICE_SERVICE_PORT=80
…
# nslookup backend-service
…
Name: backend-service
Address 1: 10.3.246.245
# nslookup backend-service
…
Name: backend-service
Address 1: 10.3.246.245
$ kubectl get svc
NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE
backend-service 10.3.246.245 <none> 80/TCP app=backend 6m
$ kubectl get svc
NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE
backend-service 10.3.246.245 <none> 80/TCP app=backend 6m

9. Internal: Kube-proxy
kube-proxykube-proxy
BB
kube-proxykube-proxy
BB
kube-proxykube-proxy
BB
Features
• TCP/UDP
• Health checks
• Client IP session affinity

10. External: NodePort
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: backend
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: backend
$ kubectl create -f backend-service-nodeport.yaml
You have exposed your service on an external port on all
nodes in your
cluster. If you want to expose this service to the
external internet, you may
need to set up firewall rules for the service port(s)
(tcp:31107) to serve traffic.
$ kubectl create -f backend-service-nodeport.yaml
You have exposed your service on an external port on all
nodes in your
cluster. If you want to expose this service to the
external internet, you may
need to set up firewall rules for the service port(s)
(tcp:31107) to serve traffic.

11. External: NodePort
Features
• TCP/UDP
• Health checks
kube-proxykube-proxykube-proxykube-proxy
BB
kube-proxykube-proxy
BB
NodePortNodePort NodePortNodePort NodePortNodePort
BB

12. External: LoadBalancer
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: backend
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
type: LoadBalancer
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: backend
$ kubectl describe svc backend-service
Name: backend-service
Namespace: default
Labels: <none>
Selector: app=backend
Type: LoadBalancer
IP: 10.3.249.155
LoadBalancer Ingress: XXX.YYY.ZZZ.III
Port: <unnamed> 80/TCP
NodePort: <unnamed> 32074/TCP
Endpoints: <none>
Session Affinity: None
$ kubectl describe svc backend-service
Name: backend-service
Namespace: default
Labels: <none>
Selector: app=backend
Type: LoadBalancer
IP: 10.3.249.155
LoadBalancer Ingress: XXX.YYY.ZZZ.III
Port: <unnamed> 80/TCP
NodePort: <unnamed> 32074/TCP
Endpoints: <none>
Session Affinity: None

13. External: LoadBalancer
Features
• TCP
• Health checks
• Client IP session affinity
(GCE)
kube-proxykube-proxykube-proxykube-proxy
BB
kube-proxykube-proxy
BB
NodePortNodePort NodePortNodePort NodePortNodePort
BB
Cloud
LB
Cloud
LB

14. External: External IPs
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
externalIPs:
- 10.240.0.2
- 10.240.0.3
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: backend
apiVersion: v1
kind: Service
metadata:
name: backend-service
spec:
externalIPs:
- 10.240.0.2
- 10.240.0.3
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: backend
$ kubectl get nodes -o json | grep -A 1 "InternalIP"
"type": "InternalIP",
"address": "10.240.0.2"
--
"type": "InternalIP",
"address": "10.240.0.3"
--
"type": "InternalIP",
"address": "10.240.0.4"
$ kubectl get nodes -o json | grep -A 1 "InternalIP"
"type": "InternalIP",
"address": "10.240.0.2"
--
"type": "InternalIP",
"address": "10.240.0.3"
--
"type": "InternalIP",
"address": "10.240.0.4"

15. External: External IPs
Features
• TCP/UDP
• Health checks
kube-proxykube-proxykube-proxykube-proxy
BB
kube-proxykube-proxy
BB BB
8080
10.240.0.2 10.240.0.3 10.240.0.4
8080

16. External: service LoadBalancer
https://github.com/kubernetes/contrib/tree/master/service-loadbalancer
1 or more HAProxy, each deployed
in a pod
Services -> HAProxy configuration
svcA-> /svcA
svcB -> /svcB
Features
• TCP/UDP, HTTP
• URL Mapping
• SSL Termination (via Annotations)
• Session Persistence (via Annotations)
• Multiple algorithms (via Annotations)

17. External: Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-ingress
spec:
rules:
- host: hello.example.com
http:
paths:
- path: /a
backend:
serviceName: backend-a
servicePort: 80
- path: /b
backend:
serviceName: backend-b
servicePort: 8080
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-ingress
spec:
rules:
- host: hello.example.com
http:
paths:
- path: /a
backend:
serviceName: backend-a
servicePort: 80
- path: /b
backend:
serviceName: backend-b
servicePort: 8080
• hello.example/a -> backend-a:80
• hello.example/b -> backend-b:8080

18. External: Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-ingress
spec:
tls:
- hosts:
- hello.example.com
secretName: hello-secret
rules:
- host: hello.example.com
. . .
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-ingress
spec:
tls:
- hosts:
- hello.example.com
secretName: hello-secret
rules:
- host: hello.example.com
. . .
apiVersion: v1
kind: Secret
metadata:
name: hello-secret
type: Opaque
data:
tls.crt: <base-64 encoded crt>
tls.key: <base-64 encoded key>
apiVersion: v1
kind: Secret
metadata:
name: hello-secret
type: Opaque
data:
tls.crt: <base-64 encoded crt>
tls.key: <base-64 encoded key>
New in 1.2: TLS support

19. External: Ingress
Features
• HTTP Load Balancing
• SSL Termination
• Content-based routing
How to use it
Ingress Controller must be deployed

20. External: Ingress
Ingress
Controller
Ingress
Controller
Ingress
Resources
Ingress
Resources
Load BalancerLoad Balancer
watches configures

21. External: Ingress
Cloud Load Balancers
• GCE HTTP Load Balancer
Software Load Balancers
• NGINX
https://github.com/kubernetes/contrib/tree/master/ingress/controllers

22. NGINX
• Layer 4/Layer 7 Load Balancer
• Advanced algorithms
• SSL termination
• Content-based routing
• Limits
• HTTP/2 gateway
• Logging
• Security
• Real-time statistics*
• Layer 7 Session Persistence*
• Dynamic reconfiguration*
* NGINX Plus
Also a webserver and cache

23. NGINX Ingress Controller
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-ingress
spec:
rules:
- host: hello.example.com
http:
paths:
- path: /a
backend:
serviceName: backend-a
servicePort: 80
- path: /b
backend:
serviceName: backend-b
servicePort: 8080
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-ingress
spec:
rules:
- host: hello.example.com
http:
paths:
- path: /a
backend:
serviceName: backend-a
servicePort: 80
- path: /b
backend:
serviceName: backend-b
servicePort: 8080
upstream backend-a {
server 10.3.246.245:80;
}
upstream backend-b {
server 10.3.246.249:8080;
}
server {
listen 80;
server_name hello.example.com;
location /a {
proxy_pass http://backend-a;
}
location /b {
proxy_pass http://backend-b;
}
}
upstream backend-a {
server 10.3.246.245:80;
}
upstream backend-b {
server 10.3.246.249:8080;
}
server {
listen 80;
server_name hello.example.com;
location /a {
proxy_pass http://backend-a;
}
location /b {
proxy_pass http://backend-b;
}
}

24. NGINX Ingress Controller
1. Watch for Ingress resources
2. Watch for Services and Endpoints: to get IP address of a service or its
endpoints in case of a headless service
3. Watch for Secrets

25. NGINX Ingress Controller
IngressIngress
EndpointsEndpoints
ServiceService
SecretSecret
IngressIngress
affects
Changes in
1. Regenerate configuration for the
Ingress
2. Reload NGINX

26. NGINX Ingress Controller
• NGINX Plus supports re-resolving DNS names in runtime every X
seconds
• Doesn’t fail when a name can’t be resolved
• Simplifies implementation: no need to watch for Services and
Endpoints

27. NGINX Ingress Controller
• As an example we took the GCE HTTP Load Balancer Ingress Controller
—
https://github.com/kubernetes/contrib/tree/master/ingress/controllers/
gce
• Written in Go
• Different implementations for NGINX and NGINX Plus
• Deployed in the same container as NGINX. the Controller starts first and
then launches NGINX.

28. NGINX Ingress Controller
• HTTP Load Balancing
• SSL Termination
• Content-based routing
Features
• Advanced algorithms
• Limits
• Access Control
• Logging
• Limits Real-time statistics (NGINX Plus)
• Layer 7 Session Persistence (NGINX Plus)
• Dynamic reconfiguration (NGINX Plus)
• and more
Features, supported by changing
NGINX templates

29. Demo
kube-proxykube-proxykube-proxykube-proxy
TT
kube-proxykube-proxy
CC
80, 44380, 443
BBCC TT

30. Demo
• tea-rc and tea-svc
• coffee-rc and headless coffee-svc
• Ingress resource cafe-ingress with TLS
• Secret cafe-secret
• NGINX Plus Ingress Controller nginx-plus-ingress-rc

31. NGINX Ingress Controller
• Expose more NGINX features via
Kubernetes resources (Annotations
and Config Maps)
• Make it production-ready
• Improve it based on your feedback
Wishlist

32. The End
● Resources: http://tiny.cc/nginx-ingress
● NGINX: https://www.nginx.com/
● My site: http://linuxjedi.co.uk/
● Twitter: @LinuxJedi
● Freenode: LinuxJedi
● Email: linuxjedi@nginx.com