Weaving Through the Mesh: Making Sense of Istio and Overlapping Technologies

1. Weaving through the Mesh Making sense of Istio and overlapping technologies Maria Gabriella Brodi @BrodiMg Cora Iberkleid @ciberkleid

2. Agenda ● Distributed systems: challenges & opportunities ● Application-based solutions ● Distributed meshes ● Broker-based solutions ● API Management solutions ● Takeaways

3. Distributed Systems

4. Distributed systems: the new normal monolith → app2app1app1 app2app2 app3 app4 app3app3 app3

5. Challenges ● Registration and discovery ● Routing and load balancing ● Fault tolerance and isolation ● Aggregation & transformation ● Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ app2app1app1 app2app2 app3 app4 app3app3 app3 ?

6. Challenges ● Registration and discovery ● Routing and load balancing ● Fault tolerance and isolation ● Aggregation & transformation ● Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ … and opportunities ● Staged rollouts ● Fault injection ● Rich metrics ● Health checks ● HTTP/2 and gRPC proxies ● TLS termination ● VMs ● Polyglot support ● AuthN-AuthZ

7. Existing and evolving approaches... ● Application-based approach ● Application-based approach with gateway ● Application-based approach with RSocket ● Service mesh approach

8. Application-based Approach

9. ● Registration and discovery ● Routing and load balancing ● Fault tolerance and isolation ● Aggregation & transformation ● Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ ?ui svcA svcB

10. discovery service ui svcA svcB lookup svcA: 192.12.12.3 svcB: 192.12.12.9 ds client ds client ds client ✓ Registration and discovery ● Routing and load balancing ● Fault tolerance and isolation ● Aggregation & transformation ● Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ register register

11. discovery service ui client lb svcA svcB lookup svcA: 192.12.12.3 svcB: 192.12.12.9 ds client ds client ds client ✓ Registration and discovery ✓ Routing and load balancing ● Fault tolerance and isolation ● Aggregation & transformation ● Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ

12. discovery service ui client lb circuit breaker svcA svcB lookup svcA: 192.12.12.3 svcB: 192.12.12.9 ds client ds client ds client ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ● Aggregation & transformation ● Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ

13. ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ● Aggregation & transformation ● Metrics & monitoring ✓ Distributed tracing ● AuthN / AuthZ discovery service ui client lb circuit breaker dt client svcA dt client svcB dt client lookup svcA: 192.12.12.3 svcB: 192.12.12.9 ds client ds client ds client

14. Solution implementation options (java examples) ● Spring Cloud ○ Distributed Configuration Load Balancing Routing Distributed Tracing ● Netflix OSS ○ Service Discovery (Eureka) Load Balancing (Ribbon) Circuit Breaker (Hystrix) ● Hashicorp Consul ○ Service Discovery Distributed Configuration Control Bus

15. Application-based Approach with Gateway

16. gateway (rte lb & cb) ● Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ✓ Aggregation & transformation ✓ Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ ui svcA svcB

17. ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ✓ Aggregation & transformation ✓ Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ ui svcA svcB gateway (rte lb & cb) discovery service ui: 192.12.12.2 svcA: 192.12.12.3 svcB: 192.12.12.9 gateway: 192.12.12.5 ds client ds client ds client

18. gateway (rte lb & cb) ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ✓ Aggregation & transformation ✓ Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ ui svcA svcB discovery service lookup ui: 192.12.12.2 svcA: 192.12.12.3 svcB: 192.12.12.9 gateway: 192.12.12.5 ds client ds client ds client

19. gateway (rte lb & cb) ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ✓ Aggregation & transformation ✓ Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ ui svcA svcB discovery service lookup lookup ui: 192.12.12.2 svcA: 192.12.12.3 svcB: 192.12.12.9 gateway: 192.12.12.5 client lb ds client ds client ds client

20. gateway (rte lb & cb) ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ✓ Aggregation & transformation ✓ Metrics & monitoring ✓ Distributed tracing ● AuthN / AuthZ ui svcA svcB lookup discovery service lookup ui: 192.12.12.2 svcA: 192.12.12.3 svcB: 192.12.12.9 gateway: 192.12.12.5 dt client dt client dt client client lb ds client ds client ds client

21. gateway (rte lb & cb) ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ✓ Aggregation & transformation ✓ Metrics & monitoring ✓ Distributed tracing ✓ AuthN / AuthZ ui svcA svcB dt client dt client dt client client lb lookup lookup auth gateway discovery service lookup ui: 192.12.12.2 svcA: 192.12.12.3 svcB: 192.12.12.9 gateway: 192.12.12.5 ds client ds client ds client

22. Solution implementation options (java examples) ● Spring Cloud ○ Distributed Configuration Load Balancing Routing Distributed Tracing ○ Gateway ● Netflix OSS ○ Service Discovery (Eureka) Load Balancing (Ribbon) Circuit Breaker (Hystrix) ○ Routing (Zuul) ● Hashicorp Consul ○ Service Discovery Distributed Configuration Control Bus

23. Advantages ● Autonomy and agility: the development team can handle these activities ● Consistency: system/domain are the same same skill set (java…)

24. Service Mesh Approach

25. svcA svcB ui ?

26. svcA svcB Sidecar Sidecar Sidecar ui

27. svcA svcB Sidecar Sidecar Sidecar ui Control Planeapply conﬁg

28. Control Plane push config push config svcA svcB Sidecar svcA:192.12.12.3 svcB:192.12.12.9 Sidecar ui:192.12.12.2 svcB:192.12.12.9 Sidecar ui:x192.12.12.2 svcA:192.12.12.3 ui apply config

29. Control Plane push conﬁg push conﬁg svcA svcB Sidecar svcA:192.12.12.3 svcB:192.12.12.9 Sidecar ui:192.12.12.2 svcB:192.12.12.9 Sidecar ui:x192.12.12.2 svcA:192.12.12.3 ui

30. Data Plane Control Plane push conﬁg push conﬁg svcA svcB Sidecar svcA:192.12.12.3 svcB:192.12.12.9 Sidecar ui:192.12.12.2 svcB:192.12.12.9 Sidecar ui:x192.12.12.2 svcA:192.12.12.3 ui

31. Data Plane Control Plane push conﬁg push conﬁg svcA svcB Sidecar svcA:192.12.12.3 svcB:192.12.12.9 Sidecar ui:192.12.12.2 svcB:192.12.12.9 Sidecar ui:x192.12.12.2 svcA:192.12.12.3 ui ✓ Registration and discovery ✓ Routing and load balancing ~ Fault tolerance and isolation ● Aggregation & transformation ✓ Metrics & monitoring ̴ Distributed tracing ✓ AuthN / AuthZ

32. Service mesh solutions ● Ingress/Egress ● East/West ● Secure communication ● Cross Cluster integration ● VMs ● Trafﬁc routing: decouple trafﬁc management from application code ● Hybrid environment: transparently to the application takes care of auth concerns ● CA management: workloads can auth between each other and across federated clusters ● Monitoring and control over latency

33. Replatforming a Gateway-based Application to Kubernetes

34. blue svc green svc yellow svc Sample App w/Gateway …on K8s Spring Cloud Gateway (rte lb & cb) lookup lookup Spring Cloud Gateway (authN/authZ) Eureka (discovery) lookup frontend premium users only SVC: LoadBalancer

35. Replatforming Steps ● Containerize & deploy each app ● Use app Service Resource names in conﬁguration ○ … for services to locate Eureka ○ … as service hostnames in Eureka registration ● Use env vars in Deployment to conﬁgure each service ○ No need to manage ports (can set server.port: 8080 for all)

36. # file: application.yml eureka: client: serviceUrl: defaultZone: http://${EUREKA_SVC}/eureka/ instance: hostname: ${CLIENT_SVC} apiVersion: apps/v1 kind: Deployment metadata: labels: app: green name: green ... containers: - image: my-reg/color-service name: green env: - name: EUREKA_SVC value: eureka - name: CLIENT_SVC value: green - name: COLOR value: green - name: SERVER_PORT value: 8080 ... apiVersion: v1 kind: Service metadata: labels: app: eureka name: eureka spec: ports: - name: eureka-8080 port: 8080 protocol: TCP targetPort: 8080 selector: app: eureka type: ClusterIP apiVersion: v1 kind: Service metadata: labels: app: green name: green spec: ports: - name: green-8080 port: 8080 protocol: TCP targetPort: 8080 selector: app: green type: ClusterIP Conﬁg

37. # file: application.yml eureka: client: serviceUrl: defaultZone: http://${EUREKA_SVC}/eureka/ instance: hostname: ${CLIENT_SVC} apiVersion: apps/v1 kind: Deployment metadata: labels: app: green name: green ... containers: - image: my-reg/color-service name: green env: - name: EUREKA_SVC value: eureka - name: CLIENT_SVC value: green - name: COLOR value: green - name: SERVER_PORT value: 8080 ... apiVersion: v1 kind: Service metadata: labels: app: eureka name: eureka spec: ports: - name: eureka-8080 port: 8080 protocol: TCP targetPort: 8080 selector: app: eureka type: ClusterIP apiVersion: v1 kind: Service metadata: labels: app: green name: green spec: ports: - name: green-8080 port: 8080 protocol: TCP targetPort: 8080 selector: app: green type: ClusterIP Conﬁg

38. Registration in Eureka Registered using K8s Service resource as hostname

39. What if you wanted to use Istio?

40. Sample App Spring Cloud Gateway (rte lb & cb) lookup lookup Spring Cloud Gateway (authN/authZ) Eureka (discovery) lookup premium users only blue svc green svc yellow svc frontend

41. Sample App with Istio Spring Cloud Gateway (rte lb & cb) lookup lookup Spring Cloud Gateway (authN/authZ) Eureka (discovery) lookup premium users only blue svc green svc yellow svc frontend

42. blue svc green svc yellow svc Sample App with Istio auth gateway frontend premium users only Ingress GW Virtual ServiceVirtual ServiceCRDs (Virtual Service Destination Rules...) Sidecar Sidecar Sidecar Sidecar Sidecar

43. Disable Eureka integrations for all apps 1. Apps should not register with Eureka 2. Apps should not attempt to look up other apps in Eureka # file: application-istio.yml eureka: client: register-with-eureka: false fetch-registry: false enabled: false

44. <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-kubernetes-ribbon</artifactId> <version>1.1.5.RELEASE</version> </dependency> # file: bootstrap-istio.yml spring.cloud.kubernetes.enabled: true # file: application-istio.yml ribbon: eureka: enabled: false http: client: enabled: true ReadTimeout: 60000 ConnectTimeout: 30000 Enable Ribbon integration with K8s for apps that call other apps (auth gateway and frontend only) Note: to use Spring Cloud Kubernetes upgrade Spring Boot and Spring Cloud (e.g Spring Boot 2.3.2.RELEASE and Spring Cloud Hoxton.SR7)

45. # File: BlueorgreengatewayApplication.java @Bean public RouteLocator routeLocator(RouteLocatorBuilder builder) { return builder.routes() .route(p -> p.path("/").or().path("/color").or().path("/js/**") .uri("lb://blueorgreenfrontend")) .route(p -> p.path("/blueorgreen") .filters(f -> f.hystrix(c -> c.setName("cmd") .setFallbackUri("forward:/colorfallback"))) .uri("lb://blueorgreen")) .build(); } Based on path, route to either frontend app or color app Now the “hard” part… translate Java into Istio YAML

46. apiVersion: networking.istio.io/v1beta1 kind: VirtualService ... http: - name: blueorgreen-svc match: - uri: prefix: /blueorgreen route: - destination: host: blueorgreen-svc port: number: 8080Based on path, route to either frontend app or color app apiVersion: networking.istio.io/v1beta1 kind: VirtualService ... http: - name: blueorgreen-fe match: - uri: prefix: / - uri: prefix: /js - uri: prefix: /color route: - destination: host: blueorgreen-fe port: number: 8080

47. # File: BlueorgreengatewayApplication.java @Bean public RouteLocator routeLocator(RouteLocatorBuilder builder) { return builder.routes() .route(p -> p.path("/").or().path("/color").or().path("/js/**") .uri("lb://blueorgreenfrontend")) .route(p -> p.path("/blueorgreen") .filters(f -> f.hystrix(c -> c.setName("cmd") .setFallbackUri("forward:/colorfallback"))) .uri("lb://blueorgreen")) .build(); } Now the “hard” part… translate Java into Istio YAML Based on path, route to either frontend app or color app if color service is slow or fails, apply a circuit breaker and call a fallback action instead

48. apiVersion: networking.istio.io/v1beta1 kind: VirtualService ... http: - name: blueorgreen-fe match: - uri: prefix: / - uri: prefix: /js - uri: prefix: /color route: - destination: host: blueorgreen-fe port: number: 8080 timeout: 1200ms apiVersion: networking.istio.io/v1beta1 kind: VirtualService ... http: - name: blueorgreen-svc match: - uri: prefix: /blueorgreen route: - destination: host: blueorgreen-svc port: number: 8080Based on path route to either frontend app or color app if color service is slow or fails apply a circuit breaker and call a fallback action instead

49. Circuit Breakers & Fallbacks With Spring Cloud Gateway… ● Sophisticated circuit breaking behavior provided through built-in Hystrix integration ● Enable centralizing configuration of circuit breakers and fallbacks With Istio... ● Simple circuit breakers (e.g. timeout) are easy to configure ● More sophisticated circuit breaking requires more involved setup ● Configuration is per Virtual Service ● Fallbacks are the responsibility of the application (not defined via Istio) Hence... (1) for this replatforming exercise the fallback definition had to be moved from the gateway app to the frontend app (2) without further effort the Istio-compatible version of the demo lacks any sophisticated circuit breaking behavior

50. # File: CustomLoadBalancerClientFilter.java (pseudocode for readability) @Override protected ServiceInstance choose(ServerWebExchange exchange) { if (<request.host = "blueorgreen">) { if (<request.cookies contains "type=premium">) { return super.choose(exchange); } else { long future = System.currentTimeMillis() + 3000; while (System.currentTimeMillis() < future) { ServiceInstance instance = super.choose(exchange); if (!instance.getMetadata().get("type").equals("premium")) { return instance; } return null; } } } return super.choose(exchange); } More of the “hard” part… translate Java into Istio YAML if the request is from a premium user call any color service; otherwise call only basic services -- services register as “premium” using Eureka metadata

51. apiVersion: networking.istio.io/v1beta1 kind: DestinationRule ... subsets: - name: basic labels: app: blueorgreen-svc access: basic - name: premium labels: app: blueorgreen-svc apiVersion: networking.istio.io/v1beta1 kind: VirtualService ... http: - name: blueorgreen-svc-premium match: - uri: prefix: /blueorgreen headers: cookie: regex: "^(.*?;)?(type=premium)(;.*)?$" route: - destination: host: blueorgreen-svc port: number: 8080 subset: premium - name: blueorgreen-svc-basic match: - uri: prefix: /blueorgreen route: - destination: host: blueorgreen-svc port: number: 8080 subset: basic if the request is from a premium user route to any color service in the premium subset; otherwise route to any service in the basic subset -- services use labels to join subsets

52. # File: ColorController.java @Value("${isSlow:false}") private boolean isSlow; @RequestMapping public Color color() throws InterruptedException { if(isSlow) { Thread.sleep(5000); } … return Color.GREEN; } # file: virtualservice.yml ... host: blueorgreen-svc subset: basic fault: delay: fixedDelay: 5000ms percentage: value: 30 Istio Fault Injections →

53. Retro (developer experience) With Spring Cloud Gateway… ● Same developer skillset ● More running applications ● Availability of debugging tools ● Redeploy after a change in the code With Istio... ● Fine grained control ● Multiple developer skillsets ● More running containers ● Debugging was challenging ● No downtime or redeploy of workloads when changing strategies ● We just scratched the surface… ● Is this the right tool/level of access for a developer? ● Who owns the Istio resources?

54. Application-based Approach with RSocket

55. RSocket Protocol Bi-directional multiplexed message-based binary protocol Based on Reactive Streams (back pressure) Transport agnostic (TCP WebSocket UDP HTTP2 …) Enables four common interaction models: ● Request-Response (1 to 1) ● Fire-and-Forget (1 to 0) ● Request-Stream (1 to many) ● Request-Channel (many to many)

56. RSocket vs HTTP - Key Differences RSocket Efficient and Responsive ● Single shared long-lived connection ● Multiplexes messages ● Communicates back pressure ● Either party can initiate requests (flexible requester/responder roles) ● Supports canceling/resuming streams HTTP Slowly Improving ● New connection per request (HTTP 1.0) ● Pipelines messages (HTTP 1.1) ● Does not communicate back pressure ● Only client can initiate requests (fixed client/server roles) ● Does not support canceling/resuming streams

62. RSocket Protocol Support Driver implementations: ● Java ● JavaScript ● Go ● .NET ● C++ ● Kotlin https://rsocket.io

63. svcA svcB rsocket

64. svcA svcB svcB svcB svcA svcA rsocket

65. ● Registration and discovery ● Routing and load balancing ● Fault tolerance and isolation ● Aggregation & transformation ● Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ svcA svcB svcB svcB svcA svcA ?

66. ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ● Aggregation & transformation ✓ Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ svcA svcB rsocket brokerrsocket client rsocket client svcB svcB svcA svcA rsocket client rsocket client rsocket client rsocket client

67. ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ● Aggregation & transformation ✓ Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ svcA svcB rsocket broker rsocket client rsocket client svcB svcB svcA svcA rsocket client rsocket client rsocket client rsocket client rsocket broker rsocket broker

68. ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ● Aggregation & transformation ✓ Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ svcA svcB rsocket broker rsocket client rsocket client svcB svcB svcA svcA rsocket client rsocket client rsocket client rsocket client rsocket broker rsocket broker

69. ✓ Registration and discovery ✓ Routing and load balancing ✓ Fault tolerance and isolation ● Aggregation & transformation ✓ Metrics & monitoring ● Distributed tracing ● AuthN / AuthZ svcA svcB rsocket broker rsocket client rsocket client svcB svcB svcA svcA rsocket client rsocket client rsocket client rsocket client rsocket broker rsocket broker S

70. RSocket Routing Broker Includes ● Broker ● Client ● Speciﬁcation Built on projects formerly known as: ● Spring Cloud RSocket ● Netiﬁ https://github.com/rsocket-routing

71. Advantages ● Autonomy and agility: the development team can handle these activities ● Consistency: system/domain are the same same skill set (java…) ● Speed ● Reduced resource utilization → savings

72. Replatforming an Rsocket Broker-based Application to Kubernetes

73. Sample App ping pong rsocket broker cluster node pong pong ping ping rsocket broker cluster node rsocket broker cluster node

74. On K8s: intra-cluster communication rsocket broker cluster node rsocket broker cluster node rsocket broker cluster node SVC: broker1 (ports 7002 & 8002) SVC: broker2 (ports 7002 & 8002) SVC: broker3 (ports 7002 & 8002)

75. apiVersion: apps/v1 kind: Deployment metadata: name: broker1 labels: app: broker instance: "1" spec: ... template: spec: containers: - image: my-reg/broker name: broker env: - name: NODE_1 value: broker2 - name: NODE_2 value: broker3 On K8s: intra-cluster communication rsocket broker cluster node rsocket broker cluster node rsocket broker cluster node SVC: broker1 (ports 7002 & 8002) SVC: broker2 (ports 7002 & 8002) SVC: broker3 (ports 7002 & 8002) # File: application.yml io.rsocket.routing.broker: tcp.port: 8002 cluster.port: 7002 brokers: - cluster: host: $NODE_1 port: 7002 proxy: host: $NODE_1 port: 8002 - cluster: host: $NODE_2 port: 7002 proxy: host: $NODE_2 port: 8002

76. # File: application.yaml io.rsocket.routing.client : service-name : ping ... brokers: - host: broker-svc port: 8002 On K8s: app to cluster communication ping pong rsocket broker cluster node pong pong ping ping rsocket broker cluster node rsocket broker cluster node SVC: broker-svc:8002

77. API Management Solutions

78. Working in front of the customer App1 AppN API Gateway & Management Auth Adapter Metrics Traffic Mgmt ... ... Certificates

79. Takeaways...

80. Takeaways... ● What’s the right tool for the right job? ○ Gateway - developer ownership and control (agility) ○ Mesh - powerful features with a nascent & evolving user experience ○ RSocket - signiﬁcant savings opportunity (e.g. edge/device connectivity) ○ API Management Solutions - address higher-level needs (e.g. external exposure of APIs) ● Tech is evolving, don’t jump too quickly ○ With Mesh in particular, tradeoff between features and ease of use ○ User experience for Mesh will improve, the mesh alone is not enough at scale ○ Let the business case be the driver for change ● Look to higher level products for enterprise needs ○ Access control at scale across roles/teams ○ Centralized operation of multiple cluster control planes

81. Other SpringOne talks to watch... Day 2 Main Stage Latest on Spring Cloud Gateway Chris Sterling Sr. Product Line Manager VMware

82. Other SpringOne talks to watch...

83. Acknowledgements & Appreciation For the insightful conversations... ● James Watters ● Dekel Tankel ● Spencer Gibb ● Chris Sterling ● Pere Monclus ● Manish Chugtu ● Jon Schneider Thank you!!

84. And thank YOU! Maria Gabriella Brodi @BrodiMg Cora Iberkleid @ciberkleid

Weaving Through the Mesh: Making Sense of Istio and Overlapping Technologies

Esté a ler uma amostra.

Confira estes a seguir

Weaving Through the Mesh: Making Sense of Istio and Overlapping Technologies

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a Weaving Through the Mesh: Making Sense of Istio and Overlapping Technologies (20)

Mais de VMware Tanzu (20)

Mais recentes (20)