Weave your microservices with Istio slide used at ATO 2019

1. Weave Your Microservices
With Istio
Lin Sun
Senior Technical Staff Member, IBM
@linsun_unc
Photo by Markos Mant on Unsplash

2. IBM Cloud
Why Service Mesh?

3. IBM Cloud
try {
HttpResponse response = httpClient.get(
“http://secretsauce.internal/recipe”);
cook(response.body);
} catch (NetworkError ne) {
fixmePleaseOMG(ne);
}
Credit to Louis Ryan for this fun example

4. IBM Cloud
try {
// Load balancing
IP ip = DNS.lookupSRV(“secretsauce.internal”).pickOne();
HttpResponse response = httpClient.open(ip).get(
“http://secretsauce.internal/recipe”);
cook(response.body);
} catch (NetworkError ne) {
fixmePleaseOMG(ne);
}
Credit to Louis Ryan for this fun example

5. IBM Cloud
for (int i = 0; i < 3; i++) { // Retry
try {
IP ip = DNS.lookupSRV(“secretsauce.internal”).pickOne();
HttpResponse response = httpClient.open(ip).get(
“http://secretsauce.internal/recipe”);
cook(response.body);
} catch (NetworkError ne) {
if (i == 2) fixmePleaseOMG(ne);
else Thread.sleep(random(5) * 1000);
}
}
Credit to Louis Ryan for this fun example

6. IBM Cloud
Secret key = new Secret(new File(“/somewhere/safe/key”);
for (int i = 0; i < 3; i++) {
try {
IP ip = DNS.lookupSRV(“secretsauce.internal”).pickOne();
HttpResponse response = httpClient.open(ip)
.setHeader(“Authorization”, key.toString())
.get(“http://secretsauce.internal/recipe”);
cook(response.body);
} catch (NetworkError ne) {
if (i == 2) fixmePleaseOMG(ne);
else Thread.sleep(random(5) * 1000);
}
}
Credit to Louis Ryan for this fun example

7. IBM Cloud
Secret key = new Secret(new File(“/somewhere/safe/key”);
for (int i = 0; i < 3; i++) {
try {
IP ip = DNS.lookupSRV(“secretsauce.internal”).pickOne();
HttpResponse response = httpClient.open(ip)
.setHeader(“Authorization”, key.toString())
.get(“http://secretsauce.internal/recipe”);
log(“Success”);
cook(response.body);
} catch (NetworkError ne) {
log(“Failed”);
if (i == 2) fixmePleaseOMG(ne);
else Thread.sleep(random(5) * 1000);
}
}
Credit to Louis Ryan for this fun example

8. IBM Cloud

9. IBM Cloud
Imagine you have
many services like
this room.
Each may use
different languages.

10. IBM Cloud
Each service owner
needs to build all
these?
Can we trust each
service owner to
build all these
consistently?

11. IBM Cloud
What exactly is
service mesh?

12. IBM Cloud

13. IBM Cloud

14. IBM Cloud
A Service Mesh is…
Language neutral Dummy initialization Program the attachment to be smartVisibility +

15. IBM Cloud
Before Service Mesh

16. IBM Cloud
Add to mesh command
Dummy initialization Visibility +

17. IBM Cloud
apply policy command
Program the attachment to be smart
mTLS
mTLS

18. IBM Cloud
Do you really need
service mesh?

19. IBM Cloud
What is Istio?
- An open service mesh platform
- Provides language neutral standard attachment to
your application container
- Provides user interfaces to configure policies for the
attachment, without redeploying your application
- Enables clear separation from the application (Dev)
and attachment (Ops)

20. IBM Cloud
data flow
management flow
From istio.io

21. IBM Cloud
Policy checksPolicy
checks
Policy Telemetry
data flow
management flow
What is missing?

22. IBM Cloud
Policy checksPolicy
checks
Policy Telemetry
Kubernetes
API server
User interactions
kubectl istioctl
data flow
management flow

23. IBM Cloud
Policy checksPolicy
checks
Policy Telemetry
Kubernetes
API server
What about auto
injection?
kubectl istioctl
Sidecar-injector
data flow
management flow

24. IBM Cloud
Policy checksPolicy
checks
Policy Telemetry
Kubernetes
API server
Mesh Boundary
kubectl istioctl
Sidecar-injector
Ingress-
gateway
Egress-
gateway
data flow
management flow

25. IBM Cloud
Install Istio
• Nothing Magic… Istio is just a bunch of CRDs, services, deployments,
config maps, secrets
• Installation Profiles
• Recommend start with the demo profile
• Use default profile as starting point for production usage

26. IBM Cloud
Deploy microservices to the mesh
• istioctl kube-inject
• kubectl label namespace {namespace} istio-injection=enabled
• istioctl add-to-mesh
• istioctl describe
https://istio.io/docs/setup/kubernetes/additional-setup/sidecar-injection/

27. IBM Cloud
Deploy pods and services to the mesh
• Add named service port for each service port
• Declare containerPort configuration for each pod
port
• Pod must have a service associated
• Label deployments with app and version
• Don't use UID 1337
• Do you have NET_ADMIN privilege?
https://istio.io/docs/setup/kubernetes/prepare/requirements/
apiVersion: v1
kind: Service
metadata:
name: productpage
labels:
app: productpage
service: productpage
spec:
ports:
- port: 9080
name: http
selector:
app: productpage
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: productpage-v1
labels:
app: productpage
version: v1
spec:
replicas: 1
template:
metadata:
labels:
app: productpage
version: v1
spec:
containers:
- name: productpage
image: istio/examples-bookinfo-
productpage-v1:1.10.1
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9080
bookinfo.yaml

28. IBM Cloud
initContainers:
- args:
- -p
- "15001"
- -u
- "1337"
- -m
- REDIRECT
- -i
- '*'
- -x
- ""
- -b
- "*"
- -d
- "15020"
image: docker.io/istio/proxy_init:1.1.0
imagePullPolicy: IfNotPresent
name: istio-init
resources:
limits:
cpu: 100m
memory: 50Mi
requests:
cpu: 10m
memory: 10Mi
securityContext:
capabilities:
add:
- NET_ADMIN
volumes:
- emptyDir:
medium: Memory
name: istio-envoy
- name: istio-certs
secret:
optional: true
secretName: istio.default
FROM ubuntu:xenial
RUN apt-get update && apt-get upgrade -y &&
apt-get install -y
iproute2
iptables
&& rm -rf /var/lib/apt/lists/*
ADD istio-iptables.sh /usr/local/bin/
ENTRYPOINT ["/usr/local/bin/istio-iptables.sh"]
echo ' -p: Specify the envoy port to which redirect all TCP traffic (default $ENVOY_PORT =
15001)'
echo ' -u: Specify the UID of the user for which the redirection is not'
echo ' applied. Typically, this is the UID of the proxy container'
# shellcheck disable=SC2016
echo ' (default to uid of $ENVOY_USER, uid of istio_proxy, or 1337)'
echo ' -g: Specify the GID of the user for which the redirection is not'
echo ' applied. (same default value as -u param)'
echo ' -m: The mode used to redirect inbound connections to Envoy, either "REDIRECT" or
"TPROXY"'
# shellcheck disable=SC2016
echo ' (default to $ISTIO_INBOUND_INTERCEPTION_MODE)'
echo ' -b: Comma separated list of inbound ports for which traffic is to be redirected to
Envoy (optional). The'
echo ' wildcard character "*" can be used to configure redirection for all ports. An
empty list will disable'
# shellcheck disable=SC2016
echo ' all inbound redirection (default to $ISTIO_INBOUND_PORTS)'
echo ' -d: Comma separated list of inbound ports to be excluded from redirection to Envoy
(optional). Only applies'
# shellcheck disable=SC2016
echo ' when all inbound traffic (i.e. "*") is being redirected (default to
$ISTIO_LOCAL_EXCLUDE_PORTS)'
echo ' -i: Comma separated list of IP ranges in CIDR form to redirect to envoy (optional).
The wildcard'
echo ' character "*" can be used to redirect all outbound traffic. An empty list will
disable all outbound'
# shellcheck disable=SC2016
echo ' redirection (default to $ISTIO_SERVICE_CIDR)'
echo ' -x: Comma separated list of IP ranges in CIDR form to be excluded from redirection.
Only applies when all '
Dockerfile.proxy_init
Istio-iptables.sh

29. IBM Cloud
- args:
- proxy
- sidecar
- --domain
- $(POD_NAMESPACE).svc.cluster.local
- --configPath
- /etc/istio/proxy
- --binaryPath
- /usr/local/bin/envoy
- --serviceCluster
- productpage.$(POD_NAMESPACE)
- --drainDuration
- 45s
- --parentShutdownDuration
- 1m0s
- --discoveryAddress
- istio-pilot.istio-system:15010
- --zipkinAddress
- zipkin.istio-system:9411
- --connectTimeout
- 10s
- --proxyAdminPort
- "15000"
- --concurrency
- "2"
- --controlPlaneAuthPolicy
- NONE
- --statusPort
- "15020"
- --applicationPorts
- “9080"
image: docker.io/istio/proxyv2:1.1.0
imagePullPolicy: IfNotPresent
name: istio-proxy
ports:
- containerPort: 15090
name: http-envoy-prom
protocol: TCP
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: INSTANCE_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: ISTIO_META_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: ISTIO_META_CONFIG_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: ISTIO_META_INTERCEPTION_MODE
value: REDIRECT
- name: ISTIO_METAJSON_LABELS
value: |
{"app":"productpage","version":"v1"}
readinessProbe:
failureThreshold: 30
httpGet:
path: /healthz/ready
port: 15020
initialDelaySeconds: 1
periodSeconds: 2
resources:
limits:
cpu: "2"
memory: 128Mi
requests:
cpu: 10m
memory: 40Mi
securityContext:
readOnlyRootFilesystem: true
runAsUser: 1337
volumeMounts:
- mountPath: /etc/istio/proxy
name: istio-envoy
- mountPath: /etc/certs/
name: istio-certs
readOnly: true
Istio-proxy container

30. IBM Cloud
Assuming you
have moved 1 or
more services to
the mesh…

31. IBM Cloud
What have you gained?
Dummy sidecar Visibility

32. IBM Cloud
$ istioctl proxy-config route productpage-v1-6597cb5df9-qlqlg --name 9080 -o json
[
…
{
"name": "9080",
"virtualHosts": [
{
{
"name": "reviews.default.svc.cluster.local:9080",
"domains": [
"reviews.default.svc.cluster.local",
"reviews.default.svc.cluster.local:9080",
"reviews",
"reviews:9080",
"reviews.default.svc.cluster",
"reviews.default.svc.cluster:9080",
"reviews.default.svc",
"reviews.default.svc:9080",
"reviews.default",
"reviews.default:9080",
"172.21.29.23",
"172.21.29.23:9080"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "outbound|9080||reviews.default.svc.cluster.local",
"timeout": "0s",
"retryPolicy": {
"retryOn": "connect-failure,refused-stream,unavailable,cancelled,resource-exhausted,retriable-status-codes",
"numRetries": 2,
"retryHostPredicate": [
{
"name": "envoy.retry_host_predicates.previous_hosts"
}
],
"hostSelectionRetryMaxAttempts": "3",
"retriableStatusCodes": [
503
]
},
"maxGrpcTimeout": "0s"
},
"decorator": {
"operation": "reviews.default.svc.cluster.local:9080/*"
},
"perFilterConfig": {
// mixer filter config
}
}
}
]
}
],
"validateClusters": false
Outbound Handler
- Routes
$ istioctl proxy-config route productpage-v1-
6597cb5df9-qlqlg --name 9080
NOTE: This output only contains routes loaded via RDS.
NAME VIRTUAL HOSTS
9080 4

33. IBM Cloud
Update: What have you gained?
Retry twiceDummy sidecar Visibility

34. IBM Cloud
Let us see it live!

35. IBM Cloud
Ready for some
intelligence?

36. IBM Cloud
Istio Network Resources
• Gateway
• Virtual Service
• Destination Rule
• Service Entry
• Envoy Filter
• Sidecar (*new*)

37. IBM Cloud
Istio Security Resources
• Authorization Policy
• Authentication Policy

38. IBM Cloud
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- route:
- destination:
host: reviews
subset: v1
Round robin is boring!
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews
trafficPolicy:
loadBalancer:
simple: RANDOM
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
- name: v3
labels:
version: v3

39. IBM Cloud
$ istioctl proxy-config route productpage-v1-6597cb5df9-qlqlg --name 9080 -o json
[
…
{
"name": "9080",
"virtualHosts": [
{
{
"name": "reviews.default.svc.cluster.local:9080",
"domains": [
"reviews.default.svc.cluster.local",
"reviews.default.svc.cluster.local:9080",
"reviews",
"reviews:9080",
"reviews.default.svc.cluster",
"reviews.default.svc.cluster:9080",
"reviews.default.svc",
"reviews.default.svc:9080",
"reviews.default",
"reviews.default:9080",
"172.21.29.23",
"172.21.29.23:9080"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "outbound|9080|v1|reviews.default.svc.cluster.local",
"timeout": "0s",
"retryPolicy": {
"retryOn": "connect-failure,refused-stream,unavailable,cancelled,resource-exhausted,retriable-status-codes",
"numRetries": 2,
"retryHostPredicate": [
{
"name": "envoy.retry_host_predicates.previous_hosts"
}
],
"hostSelectionRetryMaxAttempts": "3",
"retriableStatusCodes": [
503
]
},
"maxGrpcTimeout": "0s"
},
"decorator": {
"operation": "reviews.default.svc.cluster.local:9080/*"
},
"perFilterConfig": {
// mixer filter config
}
}
}
]
}
],
"validateClusters": false
Outbound Handler
- Routes
$ istioctl proxy-config route productpage-v1-
6597cb5df9-qlqlg --name 9080
NOTE: This output only contains routes loaded via RDS.
NAME VIRTUAL HOSTS
9080 4

40. IBM Cloud
$ istioctl pc endpoint productpage-v1-6597cb5df9-qlqlg --cluster "outbound|9080|v1|reviews.default.svc.cluster.local"
ENDPOINT STATUS CLUSTER
172.30.239.1:9080 HEALTHY outbound|9080|v1|reviews.default.svc.cluster.local
$ k get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
details-v1-bc557b7fc-hwkf4 2/2 Running 0 17d 172.30.239.62 10.188.142.197 <none>
httpbin-5fc7cf895d-8jj9r 2/2 Running 0 2d12h 172.30.177.181 10.188.142.194 <none>
productpage-v1-6597cb5df9-qlqlg 2/2 Running 0 17d 172.30.177.159 10.188.142.194 <none>
ratings-v1-5c46fc6f85-gqb8p 2/2 Running 0 17d 172.30.177.175 10.188.142.194 <none>
reviews-v1-69dcdb544-6rdff 2/2 Running 0 17d 172.30.239.1 10.188.142.197 <none>
reviews-v2-65fbdc9f88-zx6fx 2/2 Running 0 17d 172.30.177.177 10.188.142.194 <none>
reviews-v3-bd8855bdd-dndgk 2/2 Running 0 17d 172.30.239.63 10.188.142.197 <none>
sleep-64c6f57bc8-f5n4x 2/2 Running 0 29d 172.30.177.144 10.188.142.194 <none>
Outbound Handler - Endpoint

41. IBM Cloud
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- match:
- headers:
end-user:
exact: jason
route:
- destination:
host: reviews
subset: v2
- route:
- destination:
host: reviews
subset: v1
Let’s A/B test reviews v2
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews
spec:
host: reviews
trafficPolicy:
loadBalancer:
simple: RANDOM
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
- name: v3
labels:
version: v3

42. IBM Cloud
$ istioctl proxy-config route productpage-v1-6597cb5df9-qlqlg --name 9080 -o json
…
{
"name": "reviews.default.svc.cluster.local:9080",
"domains": [
"reviews.default.svc.cluster.local",
"
],
"routes": [
{
"match": {
"prefix": "/",
"headers": [
{
"name": "end-user",
"exactMatch": "jason"
}
]
},
"route": {
"cluster": "outbound|9080|v2|reviews.default.svc.cluster.local",
"timeout": "0s",
"retryPolicy": {
"retryOn": "connect-failure,refused-stream,unavailable,cancelled,resource-exhausted,retriable-status-codes",
"numRetries": 2,
"retryHostPredicate": [
{
"name": "envoy.retry_host_predicates.previous_hosts"
}
],
"hostSelectionRetryMaxAttempts": "3",
"retriableStatusCodes": [
503
]
},
"maxGrpcTimeout": "0s"
},
…
},
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "outbound|9080|v1|reviews.default.svc.cluster.local",
"timeout": "0s",
"retryPolicy": {
"retryOn": "connect-failure,refused-stream,unavailable,cancelled,resource-exhausted,retriable-status-codes",
"numRetries": 2,
"retryHostPredicate": [
{
"name": "envoy.retry_host_predicates.previous_hosts"
}
],
"hostSelectionRetryMaxAttempts": "3",
"retriableStatusCodes": [
503
]
},
"maxGrpcTimeout": "0s"
},
…
}
]
}
Outbound Handler
- Routes
$ istioctl proxy-config route productpage-v1-
6597cb5df9-qlqlg --name 9080
NOTE: This output only contains routes loaded via RDS.
NAME VIRTUAL HOSTS
9080 4

43. IBM Cloud
$ istioctl pc endpoint productpage-v1-6597cb5df9-qlqlg --cluster "outbound|9080|v1|reviews.default.svc.cluster.local"
ENDPOINT STATUS CLUSTER
172.30.239.1:9080 HEALTHY outbound|9080|v1|reviews.default.svc.cluster.local
$ istioctl pc endpoint productpage-v1-6597cb5df9-qlqlg --cluster "outbound|9080|v2|reviews.default.svc.cluster.local"
ENDPOINT STATUS CLUSTER
172.30.177.177:9080 HEALTHY outbound|9080|v2|reviews.default.svc.cluster.local
$ k get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
details-v1-bc557b7fc-hwkf4 2/2 Running 0 17d 172.30.239.62 10.188.142.197 <none>
httpbin-5fc7cf895d-8jj9r 2/2 Running 0 2d12h 172.30.177.181 10.188.142.194 <none>
productpage-v1-6597cb5df9-qlqlg 2/2 Running 0 17d 172.30.177.159 10.188.142.194 <none>
ratings-v1-5c46fc6f85-gqb8p 2/2 Running 0 17d 172.30.177.175 10.188.142.194 <none>
reviews-v1-69dcdb544-6rdff 2/2 Running 0 17d 172.30.239.1 10.188.142.197 <none>
reviews-v2-65fbdc9f88-zx6fx 2/2 Running 0 17d 172.30.177.177 10.188.142.194 <none>
reviews-v3-bd8855bdd-dndgk 2/2 Running 0 17d 172.30.239.63 10.188.142.197 <none>
sleep-64c6f57bc8-f5n4x 2/2 Running 0 29d 172.30.177.144 10.188.142.194 <none>
Outbound Handler - Endpoint

44. IBM Cloud
More Sidecar Debug
$ istioctl dashboard envoy $(kubectl get pod -l
app=productpage -o
jsonpath='{.items[0].metadata.name}')
http://localhost:56740

45. IBM Cloud
How many lines are my
envoy configuration?

46. IBM Cloud
Program
Envoy is hardProgram envoy is hard

47. IBM Cloud
Program
Envoy is hard

48. IBM Cloud
Let us see it live!

49. IBM Cloud
2019 Istio Themes
• Project Sustainability
• Layering and Extensibility
• Improved Experience
• Performance and Scalability

50. IBM Cloud
Istio 2019 Predictable Releases
Istio
1.0
Launch
-July
2018
2018 2019
Istio
1.1
February
Istio
1.2
June
Istio
1.3
Septem
ber
2020
Istio
1.4
Unreleased

51. IBM Cloud
2019 Istio Key Updates
• All outbound traffic is allowed by default
• Mixer-policy is not enabled by default
• Multicluster Istio for non flat networks
• Introduce Sidecar resource
• Intelligent Protocol Sniffing
• Various istioctl UX improvements

52. IBM Cloud
How to contribute
• Contribute via discuss or slack
• Gain membership Status
• Consult existing maintainers
• Gain maintainer status
• Attend WG Meetings

53. IBM Cloud
• Preview available around
KubeCon US 2019
• Final book available by
end of 2019