Within complex systems, even well-written code can behave in unexpected ways and lead to outages and critical issues. Amazon Elastic Container Service for Kubernetes (Amazon EKS) enables you to easily run Kubernetes, quickly deploy new code, and revert to safe, stable releases when issues are identified. But the damage done in the short period between deployment and rollback can be significant. In this session, we show you how to limit the effect of unforeseen issues using canary deployments with Istio and how to better monitor your applications in Amazon EKS and spot potential problems before they affect your customer base. This session is brought to you by AWS partner, Datadog.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved.
Jason Yee
Technical Evangelist, Datadog
SRV305
Canary Deploys on Amazon EKS with Istio

2. C A N A R Y D E P L O Y S O N A M A Z O N
E K S W I T H I S T I O
J A S O N Y E E , T E C H N I C A L E V A N G E L I S T , D A T A D O G

3. D E P L O Y M E N T S
W H A T C O U L D P O S S I B L Y G O W R O N G ?

4. A L O T !
D U H .

5. When I deploy to prod… I do it in production.

6. When I deploy to prod… I do it in production.

7. When I deploy to prod… I do it in production.
# Y O L O o p s

8. C A N A R Y D E P L O Y S
W I T H K U B E R N E T E S A N D I S T I O

9. J A S O N Y E E
T e c h n i c a l E v a n g e l i s t
C o l l e c t o r o f :
• M i l e s
• W h i s k e y
• P o k e m o n
T w : @ g i t b i s e c t
E m : j y e e @ d a t a d o g h q . c o m

10. D A T A D O G
T W : @ d a t a d o g h q
S a a S - b a s e d m o n i t o r i n g , t r a c i n g
& l o g g i n g
T r i l l i o n s o f p o i n t s / d a y
C a v e a t : W e r u n s o m e
K u b e r n e t e s i n p r o d u c t i o n & a r e
e x p l o r i n g p a r t s o f I s t i o

11. B L U E - G R E E N D E P L O Y M E N T S

14. B L U E - G R E E N D E P L O Y M E N T S
• Pros:
• Zero-downtime deploys
• Easy rollbacks

15. C T R L + Z

16. B L U E - G R E E N D E P L O Y M E N T S
• Cons:
• "Easy" rollbacks… 😬

17. R O L L I N G
D E P L O Y M E N T S

23. C A N A R Y D E P L O Y M E N T S

24. or

27. P A U S E . M O N I T O R .

29. D O U B L E C H E C K .

31. M A Y B E O N E M O R E T I M E , J U S T
T O B E C E R T A I N .

33. P A R T Y !

34. C A N A R Y D E P L O Y M E N T S
• Small scope

35. C A N A R Y D E P L O Y M E N T S
• Small scope
• Limited ramifications

36. C A N A R Y D E P L O Y M E N T S
• Small scope
• Limited ramifications
• Easier rollbacks

37. C A N A R Y D E P L O Y M E N T S
• Small scope
• Limited ramifications
• Easier rollbacks
• Load tolerant

38. C A N A R Y D E P L O Y M E N T S
• Small scope
• Limited ramifications
• Easier rollbacks
• Load tolerant
• Concurrency

39. C A N A R Y
S T R A T E G Y

40. C A N A R Y S T R A T E G Y
H o w d o y o u c h o o s e y o u r s a m p l e s e t ?
• Random

41. C A N A R Y S T R A T E G Y
H o w d o y o u c h o o s e y o u r s a m p l e s e t ?
• Random
• Representative

42. C A N A R Y S T R A T E G Y
H o w d o y o u c h o o s e y o u r s a m p l e s e t ?
• Random
• Representative
• Geography

43. C A N A R Y S T R A T E G Y
H o w d o y o u c h o o s e y o u r s a m p l e s e t ?
• Random
• Representative
• Geography
• Time

44. C A N A R Y S T R A T E G Y
H o w d o y o u c h o o s e y o u r s a m p l e s e t ?
• Random
• Representative
• Geography
• Time
• Use patterns

45. C A N A R Y S T R A T E G Y
H o w d o y o u c h o o s e y o u r s a m p l e s e t ?
• Random
• Representative
• Geography
• Time
• Use patterns
• Granularity

46. C A N A R Y S T R A T E G Y
H o w d o y o u c h o o s e y o u r s a m p l e s e t ?
• Random
• Representative
• Geography
• Time
• Use patterns
• Granularity
• Resource mapping

47. M O N I T O R I N G
S T R A T E G Y

48. M O N I T O R I N G S T R A T E G Y
H o w d o y o u e v a l u a t e y o u r d e p l o y m e n t ?
• Tags! Tags! Tags! Tags! Tags!

51. M O N I T O R I N G S T R A T E G Y
H o w d o y o u e v a l u a t e y o u r d e p l o y m e n t ?
• Tags!
• p90, p95, p99

52. M O N I T O R I N G S T R A T E G Y
H o w d o y o u e v a l u a t e y o u r d e p l o y m e n t ?
• Tags!
• p90, p95, p99
• Outliers

54. Outliers: one of these things is not like the others

55. M O N I T O R I N G S T R A T E G Y
H o w d o y o u e v a l u a t e y o u r d e p l o y m e n t ?
• Tags!
• p90, p95, p99
• Outliers
• Anomalies

56. Anomalies: It wasn’t like this before

57. S I G N A L S T O W A T C H
Latency

58. S I G N A L S T O W A T C H
Latency Errors

59. S I G N A L S T O W A T C H
Latency Errors
Traffic

60. S I G N A L S T O W A T C H
Latency Errors
Traffic Saturation

61. W H A T D O E S K U B E R N E T E S H A V E T O
D O W I T H A N Y O F T H I S ?

62. C O N T A I N E R S E R V I C E
O R C H E S T R A T O R

63. C O N T A I N E R S E R V I C E
O R C H E S T R A T O R
p.s. - Maybe a Squirtle orchestrator? Talk to me later if you want a sticker.

64. So what if we want to deploy one service?

65. Blue-green doesn’t make any sense!

66. Kubernetes handles service deployments

67. Kubernetes handles service deployments. YAY!

68. W H Y D O I N E E D A
S E R V I C E M E S H ?

69. K u b e r n e t e s d o e s
r o l l i n g d e p l o y s
r e a l l y w e l l !

70. C a n a r y d e p l o y s ,
n o t s o m u c h .

71. C A N A R Y D E P L O Y I N G W I T H K U B E R N E T E S

72. S E R V I C E
apiVersion: v1
kind: Service
metadata:
name: process
labels:
app: process
spec:
ports:
- port: 80
name: http
selector:
app: process

73. D E P L O Y M E N T
apiVersion: apps/v1
kind: Deployment
metadata:
name: process
labels:
app: process
spec:
replicas: 3
selector:
matchLabels:
app: process
template:
metadata:
labels:
app: process
version: 1.0
spec:
containers:
- name: pull
image: jyee/process:v1
imagePullPolicy: Always

74. D E P L O Y M E N T
apiVersion: apps/v1
kind: Deployment
metadata:
name: process
labels:
app: process
spec:
replicas: 3
selector:
matchLabels:
app: process
template:
metadata:
labels:
app: process
version: 1.0
spec:
containers:
- name: pull
image: jyee/process:v1
imagePullPolicy: Always
apiVersion: apps/v1
kind: Deployment
metadata:
name: process
labels:
app: process
spec:
replicas: 3
selector:
matchLabels:
app: process
template:
metadata:
labels:
app: process
version: 2.0
spec:
containers:
- name: pull
image: jyee/process:v2
imagePullPolicy: Always

75. D E P L O Y M E N T
apiVersion: apps/v1
kind: Deployment
metadata:
name: process
labels:
app: process
spec:
replicas: 9
selector:
matchLabels:
app: process
template:
metadata:
labels:
app: process
version: 1.0
spec:
containers:
- name: pull
image: jyee/process:v1
imagePullPolicy: Always
apiVersion: apps/v1
kind: Deployment
metadata:
name: process
labels:
app: process
spec:
replicas: 1
selector:
matchLabels:
app: process
template:
metadata:
labels:
app: process
version: 2.0
spec:
containers:
- name: pull
image: jyee/process:v2
imagePullPolicy: Always

76. D E P L O Y M E N T
apiVersion: apps/v1
kind: Deployment
metadata:
name: process
labels:
app: process
spec:
replicas: 8
selector:
matchLabels:
app: process
template:
metadata:
labels:
app: process
version: 1.0
spec:
containers:
- name: pull
image: jyee/process:v1
imagePullPolicy: Always
apiVersion: apps/v1
kind: Deployment
metadata:
name: process
labels:
app: process
spec:
replicas: 2
selector:
matchLabels:
app: process
template:
metadata:
labels:
app: process
version: 2.0
spec:
containers:
- name: pull
image: jyee/process:v2
imagePullPolicy: Always

77. W H A T D O E S A S E R V I C E M E S H G E T Y O U ?

78. S E R V I C E M E S H E S
• Routing & load balancing

79. S E R V I C E M E S H E S
• Routing & load balancing
• Service discovery

80. S E R V I C E M E S H E S
• Routing & load balancing
• Service discovery
• Timeouts & retries

81. S E R V I C E M E S H E S
• Routing & load balancing
• Service discovery
• Timeouts & retries
• Policy enforcement

82. S E R V I C E M E S H E S
• Routing & load balancing
• Service discovery
• Timeouts & retries
• Policy enforcement
• Monitoring & tracing

83. How does it work?

84. 1. Add a data plane

86. 2. Add a control plane

88. 1 . 0 i s c o m i n g .
G o p l a y w i t h I s t i o

89. 1 . 0 i s c o m i n g .
G o p l a y w i t h I s t i o
HERE! 🎉

90. b u t d o n ’ t r o l l i t o u t i n p r o d

91. b u t d o n ’ t r o l l i t o u t i n p r o d
TRY TO

92. I N S T A L L I N G I S T I O
1. Download from github.com/istio/istio/releases

93. I N S T A L L I N G I S T I O
1. Download from github.com/istio/istio/releases
2. Unpack

94. I N S T A L L I N G I S T I O
1. Download from github.com/istio/istio/releases
2. Unpack
3. export PATH=$PATH:istio-1.0.0/bin

95. I N S T A L L I N G I S T I O
1. Download from github.com/istio/istio/releases
2. Unpack
3. export PATH=$PATH:istio-1.0.0/bin
4. kubectl apply -f install/kubernetes/istio-demo.yaml

96. I N S T A L L I N G I S T I O
1. Download from github.com/istio/istio/releases
2. Unpack
3. export PATH=$PATH:istio-1.0.0/bin
4. kubectl apply -f install/kubernetes/istio-demo.yaml
Or Use Helm!

97. W H A T D O E S I T I N S T A L L ?
Istio Citadel
Istio Egress Gateway
Istio Ingress Gateway
Istio Pilot
Istio Policy
Istio Sidecar Injector
Istio Telemetry
Statsd Prometheus Bridge
Grafana
Prometheus
Service Graph NG
Zipkin

98. C A N A R Y D E P L O Y I N G W I T H I S T I O

99. S E R V I C E
apiVersion: v1
kind: Service
metadata:
name: process
labels:
app: process
spec:
ports:
- port: 80
name: http
selector:
app: process

100. D E P L O Y M E N T
apiVersion: apps/v1
kind: Deployment
metadata:
name: process
labels:
app: process
spec:
replicas: 3
selector:
matchLabels:
app: process
template:
metadata:
labels:
app: process
version: 1.0
spec:
containers:
- name: pull
image: jyee/process:v1
imagePullPolicy: Always
apiVersion: apps/v1
kind: Deployment
metadata:
name: process
labels:
app: process
spec:
replicas: 3
selector:
matchLabels:
app: process
template:
metadata:
labels:
app: process
version: 2.0
spec:
containers:
- name: pull
image: jyee/process:v2
imagePullPolicy: Always

101. k u b e c t l c r e a t e - f
< ( i s t i o c t l k u b e - i n j e c t - f y o u r _ k 8 s . y a m l )

102. T E E N A G E
M U T A T I N G W E B H O O K
A D M I S S I O N C O N T R O L L E R S !
A K A A U T O - S I D E C A R I N J E C T I O N

103. C O N T A I N E R S E R V I C E
O R C H E S T R A T O R

104. C O N T A I N E R S E R V I C E
O R C H E S T R A T O R M E S H

105. I S T I O V I R T U A L S E R V I C E S
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: process-routing
spec:
hosts:
- process
http:
- route:
- destination:
host: process
subset: v1

106. I S T I O V I R T U A L S E R V I C E S
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: destination-process
spec:
host: process
subsets:
- name: v1
labels:
version: '1.0'

107. I S T I O V I R T U A L S E R V I C E S
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: destination-process
spec:
host: process
subsets:
- name: v1
labels:
version: '1.0'
- name: v2
labels:
version: '2.0'

108. I S T I O V I R T U A L S E R V I C E S
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: destination-process
spec:
host: process
subsets:
- name: v1
labels:
version: '1.0'
- name: v2
labels:
version: '2.0'

109. I S T I O V I R T U A L S E R V I C E S
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: process-routing
spec:
hosts:
- process
http:
- route:
- destination:
host: process
subset: v1
weight: 80
- route:
- destination:
host: process
subset: v2
weight: 20

110. I S T I O V I R T U A L S E R V I C E S
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: process-routing
spec:
hosts:
- process
http:
- route:
- destination:
host: process
subset: v1
weight: 80
- route:
- destination:
host: process
subset: v2
weight: 20

111. I S T I O V I R T U A L S E R V I C E S
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
...
http:
- match:
- headers:
cookie:
user: my-logged-in-user
route:
- destination:
host: process
subset: v2
weight: 20

112. L O T S !
W H A T E L S E C A N I T D O ?
h t t p s : / / i s t i o . i o / d o c s / r e f e r e n c e / c o n f i g /

113. R E C A P
• Service meshes give you more control

114. R E C A P
• Service meshes give you more control
• Canary deploys: Representative & Granular

115. R E C A P
• Service meshes give you more control
• Canary deploys: Representative & Granular
• Monitoring: Tags, Outliers, Anomalies

116. R E C A P
• Service meshes give you more control
• Canary deploys: Representative & Granular
• Monitoring: Tags, Outliers, Anomalies
• What to watch: Latency, Errors, Traffic, Saturation

117. R E C A P
• Service meshes give you more control
• Canary deploys: Representative & Granular
• Monitoring: Tags, Outliers, Anomalies
• What to watch: Latency, Errors, Traffic, Saturation
• GO PLAY WITH ISTIO 1.0.0!

118. Q U E S T I O N S ?
e m a i l : j y e e @ d a t a d o g h q . c o m
t w i t t e r : @ g i t b i s e c t

119. S L I D E S : b i t . l y / a w s c h i - c a n a r i e s
e m a i l : j y e e @ d a t a d o g h q . c o m
t w i t t e r : @ g i t b i s e c t

120. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Submit session feedback
1. Tap the Schedule icon.
2. Select the session you attended.
3. Tap Session Evaluation to submit your
feedback.