The document discusses common DevOps challenges related to rolling out new versions of microservices and testing them. It introduces Istio as a solution for addressing these challenges through intelligent routing, resiliency features, traffic controls, telemetry collection, and other capabilities. Istio uses the Envoy proxy and control tools like Pilot and Mixer to provide features for reliable traffic management between services, such as advanced routing rules for canary releases, fault injection for testing resiliency, and policy enforcement across the mesh.
1. Reliable application roll out and
operations with Istio
Lin Sun, IBM @linsun_unc
Mandar Jog, Google @mandarjog
2. Common DevOps Challenge 1
• How do I roll out a newer version of my
microservice without down time?
• How do I ensure traffic continue goes
to the current version before the newer
version is tested and ready?
3. Common DevOps Challenge 2
• How do I do A/B testing?
• Release a new version to a
subset of users in a precise way
• I have launched B in the dark,
but how can I keep B to myself
or a small testing group?
4. Common DevOps Challenge 3
• How do I do canary testing?
• I want to leverage crowdsourced
testing. How do I test the new
version to a subset of users?
• How do I proceed to a full rollout
after satisfactory testing of the new
version?
5. Other Common DevOps Challenges
• Things don’t always go correctly in production…
How do I inject fault to my microservices to
prepare myself?
• Our team knows different languages and our
services are written in different languages.
• My services can only handle certain rate, how
can I limit rate for some of my services?
• I need to view what is going on with each of my
services when crisis arises.
11. Components of Istio
• Envoy proxy, to mediate all inbound and outbound traffic for all services in the service mesh.
Leverages Envoy features such as dynamic service discovery, load balancing, TLS
termination, HTTP/2 & gRPC proxying, circuit breakers, health checks, staged rollouts with %-
based traffic split, fault injection, and rich metrics.
• Pilot: Programming envoys and responsible for service discovery, registration and load
balancing
• Istio-Security provides strong service-to-service and end-user authentication using mutual
TLS, with built-in identity and credential management
• Mixer is responsible for enforcing access control and usage policies across the service mesh
and collecting telemetry data from the Envoy proxy and other services.
14. Traffic Control
// A simple traffic control rule
destination:
name: serviceB.example.cluster.local
match:
source: serviceA.example.cluster.local
route:
- labels:
version: v1.5
env: us-prod
weight: 100
Challenge 1: How can I roll out new version without
down time or changing code?
15. Traffic Steering
// Content-based traffic steering rule
destination:
serviceB.example.cluster.local
match:
httpHeaders:
user-agent:
regex: ^(.*?;)?(iPhone)(;.*)?$
precedence: 2
route:
- labels:
version: v2
Challenge 2: How do I do A/B testing?
16. Traffic Splitting
// A simple traffic splitting rule
destination:
serviceB.example.cluster.local
match:
source:
serviceA.example.cluster.local
route:
- labels:
version: v1.5
env: us-prod
weight: 90
- labels:
version: v2.0-alpha
env: us-staging
weight: 10
Challenge 3: How do I do canary testing?
17. Resiliency
// Circuit breakers
destination: serviceB.example.cluster.local
policy:
- labels:
version: v1
circuitBreaker:
simpleCb:
maxConnections: 100
httpMaxRequests: 1000
httpMaxRequestsPerConnection: 10
httpConsecutiveErrors: 7
sleepWindow: 15m
httpDetectionInterval: 5m
Istio adds fault tolerance to your application
without any changes to code Resilience features
❖ Timeouts
❖ Retries with timeout budget
❖ Circuit breakers
❖ Health checks
❖ AZ-aware load balancing w/ automatic
failover
❖ Control connection pool size and request
load
20. Telemetry
Monitoring & tracing should not be an
afterthought in the infrastructure
Goals
● Metrics without instrumenting apps
● Consistent metrics across fleet
● Trace flow of requests across services
● Portable across metric backend
providers