1. @abhishektiwari
Kubernetes sidecar
pattern as a swiss-army
knife for microservices
Abhishek Tiwari
https://www.abhishek-tiwari.com

2. @abhishektiwari
A bit about me
● Director of Engineering at HelloFresh
● An early adopter of K8S ~ Dec 2015
● Ran large K8s clusters in AWS and GCP
● A range of mission critical stateles workloads

3. @abhishektiwari
scaling microservices require excellence in devops
Amazon Twitter

4. @abhishektiwari
J-CURVE OF DevOPs Excellence
Credits: Accelerate: State of DevOps 2018: Strategies for a New Economy | Does DevOps Matter?
7% Elite

5. @abhishektiwari
Common DevOps Concerns
A
B
C
Service-to-service
communication
A
B
C
Securing services and
communication
A
B
C
Control and enforce
policies

6. @abhishektiwari
Common DevOps Concerns
A
B
C
Service observability
and telemetry
A
B
C
Fault tolerance and
circuit breakers
A
B
C
Deployments and
service topologies

7. @abhishektiwari
Old Approach
fault tolerance libraries
Implementation specific to
- Languages (Java/Scala) or
- Frameworks or
- Server (Tomcat/Jetty)or
- Protocols (Thrift/RPC)

8. @abhishektiwari
Kubernetes
Kubernetes has now become the de facto standard
for deploying containerized applications at scale
in private, public and hybrid cloud.

9. @abhishektiwari
High-level architecture
Kubernetes
Master
Node Node Node
Pod Pod
Pod Pod
Pod
Pod
Pod
Pod
Pod
Pod
Pod Pod
Pods are scheduled and packed dynamically on Kubernetes nodes
Docker Kubelet Kube Proxy Docker Kubelet Kube Proxy Docker Kubelet Kube Proxy

10. @abhishektiwari
PODS
A pod can co-schedule multiple containers as an atomic unit.
MySQL
Django
Nginx
MySQL
Django
Nginx
Co-scheduled multiple
containers as pod
Scheduled independently as
containers

11. @abhishektiwari
Design patterns for
container-based
distributed systems

12. @abhishektiwari
Design patterns for
container-based
distributed systems
3 Essential
Patterns
● Single-pod single-container patterns
● Single-pod multiple-container patterns
● multi-pod patterns

13. @abhishektiwari
MySQL
Django
Nginx
MySQL
Django
Nginx
Single-pod, multiple-
containers pattern
Single-pod, single-
container pattern
PODS
MySQL
Django
Nginx
1 2
3
Combination of 1 & 2

14. @abhishektiwari
MySQL
3
Combination of 1 & 2
Django
Nginx
Django
Nginx
Stateless Autoscaling of PODS
Django
Nginx

15. @abhishektiwari
Main container
Sidecar container
Sidecar pattern
A sidecar is a utility container
in the Pod and its whole purpose
is to support the main container
Fluentd
Python App
error.log

16. @abhishektiwari
● Independent resource
● Completely reusable
● Graceful degradation
● Seperate life cycle
● Runtime injection
● Multiple per main
● Peripheral tasks
Benefits of
Sidecar

17. @abhishektiwari
Envoy Linkerd
Sidecar proxy (aka data plane)
traefik
Intelligent service proxy which mediate
and/or control all network communication
Nginx
HAProxy

18. @abhishektiwari
Nginx Sidecar proxy
MySQL
Django
Nginx
Sidecar Nginx proxy mediates
all traffic to and from main
Django container
Python App

19. @abhishektiwari
Service to service communication
Service-A
Envoy
Service-B
Envoy
Service-C
Envoy
Envoy.yaml: Routing
virtual_hosts:
- name: backend
domains:
- "*"
routes:
- match:
prefix: "/service/a"
route:
cluster: service_a
- match:
prefix: "/service/b"
route:
cluster: service_b
- match:
prefix: "/service/c"
route:
cluster: service_c

20. @abhishektiwari
Service to service communication
Service-A
Envoy
Service-B
Envoy
Service-C
Envoy
Envoy.yaml: Load Balancing
clusters:
- name: service_a
connect_timeout: 0.25s
type: strict_dns
lb_policy: round_robin
http2_protocol_options: {}
hosts:
- socket_address:
address: service_a
port_value: 443
- name: service_b
connect_timeout: 0.25s
type: strict_dns
lb_policy: round_robin
http2_protocol_options: {}
hosts:
- socket_address:
address: service_b
port_value: 443

21. @abhishektiwari
Securing services and communication
Service-A
Envoy
Service-B
Envoy
Service-C
Envoy
Envoy.yaml: JWT Authentication
providers:
jwt_provider1:
issuer: https://auth0.com
audiences:
audience1
local_jwks:
inline_string: PUBLIC-KEY
rules:
- match:
prefix: /health
- match:
prefix: /api
requires:
provider_and_audiences:
provider_name: jwt_provider1
audiences:
api_audience
- match:
prefix: /
requires:
provider_name: jwt_provider1

22. @abhishektiwari
Fault tolerance and circuit breakers
Service-A
Envoy
Service-B
Envoy
Service-C
Envoy
Envoy.yaml: Circuit Breakers
circuit_breakers:
thresholds:
max_connections: 1
max_pending_requests: 1
max_requests: 1

23. @abhishektiwari
Fault tolerance and circuit breakers
Service-A
Envoy
Service-B
Envoy
Service-C
Envoy
Envoy.yaml: Retry/Timeout
retry_policy:
retry_on: 5xx
num_retries: 3
per_try_timeout: 5s

24. @abhishektiwari
Fault tolerance and circuit breakers
Service-A
Envoy
Service-B
Envoy
Service-C
Envoy
Envoy.yaml: Fault/Delay
http_filters:
- name: envoy.fault
config:
delay:
type: fixed
fixed_delay: 10s
percentage:
numerator: 50
denominator: HUNDRED

25. @abhishektiwari
Service observability and telemetry
Service-A
Envoy
Service-B
Envoy
Service-C
Envoy
Envoy.yaml: Zipkin Tracing
tracing:
http:
name: zipkin
typed_config:
type: zipkin
collector_cluster: zipkin
collector_endpoint: "/zipc"

26. @abhishektiwari
Configuration hell
Static
configs
Envoy
Envoy
Envoy

27. @abhishektiwari
We need a control plane
Control Plane
Manages and configures the proxies, enforce policies and collect telemetry
Service-A
Envoy
Service-B
Envoy
Service-C
Envoy
Data Plane

28. @abhishektiwari
Control plane + Data Plane = Service mesh

29. @abhishektiwari
Lastly
What microservices are part of my
service mesh and how are
they connected?

30. @abhishektiwari
THanks
Q&A