O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Gluecon - Kafka and the service mesh

3.373 visualizações

Publicada em

Exploring the problem of Microservices communication and how both Kafka and Service Mesh solutions address it. We then look at some approaches for combining both.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Gluecon - Kafka and the service mesh

  1. 1. 1 Kafka and the Service Mesh Gwen Shapira, Principal Data Architect @gwenshap
  2. 2. You Have Microservices 2
  3. 3. They need to communicate 3
  4. 4. 4 Orders Returns ? Invento ry Fulfill Order Validat e Order I know! I’ll use REST APIs
  5. 5. Synchronous request-response communication Leads to Tight point-to-point coupling 5
  6. 6. 6 Making Changes is Risky Change
  7. 7. 7 Adding Services Requires Explicit Calls Mistak e Handle r
  8. 8. All these lines of communication require: Logging, monitoring, error handling and more 8
  9. 9. The most difficult thing about microservices is data. Especially joining data. 9
  10. 10. We need a way to communicate that is: Decoupled Flexible Operationally Transparent Data Aware 10
  11. 11. Kafka? 11
  12. 12. 15 From This:
  13. 13. 16 To This:
  14. 14. 17 And This: New Servi ce
  15. 15. Kafka: - Immutable, ordered, stream (log) of events - Highly Available - Scalable - Load balancing consumers - HA consumers - (Potentially) infinite storage
  16. 16. Kafka + Schema Registry 20
  17. 17. APIs between services are Contracts In Stream Processing World – Event Schemas ARE the API …except they stick around for a lot longer
  18. 18. We need a way to communicate that is: Decoupled Flexible Operationally Transparent Data Aware 27
  19. 19. Service Mesh? 28
  20. 20. It is a proxy.
  21. 21. Sidecar: Components of the application, deployed in a separate container to provide isolation and encapsulation. This pattern allows applications to be composed of heterogeneous components.
  22. 22. 31
  23. 23. Proxy as sidecar: Proxy Proxy Proxy Proxy Proxy
  24. 24. Proxy I have a new IP now. YOLO! Who cares? I magically know all about it!
  25. 25. Proxy I can recover from errors without drowning Error? No worries! Lets retry every millisecond forever LOL. I’m dropping 99% of the retries.
  26. 26. Cool Things proxies do! - One stable uri for each service - Service discovery - Log requests - Monitor request latency - Routing - A/B testing, green/blue deployments - Circuit breaking - Protocol translation - SSL Termination
  27. 27. They don’t do it alone
  28. 28. We need a way to communicate that is: Decoupled Flexible Operationally Transparent Data Aware 37
  29. 29. How do we choose? 38
  30. 30. 40 Two different paradigms Request-Response Events Streams - Very Low latency - Often sync - Point-to-point - “Bespoke API” - Continuous processing - Often async - Event driven - General-purpose events
  31. 31. Vision #1: Using Service Mesh to hide Kafka Introduce
  32. 32. Proxy I am somehow getting events from Kafka I’m using REST to talk to a service I’m really re- directing events to Kafka
  33. 33. REST Proxy does bits of that. HTTP on one-side, Kafka on the other. We are adding more protocols: - JMS - MQTT - gRPC
  34. 34. Vision #2: Kafka as part of control plane
  35. 35. Proxy I am using REST too! Kafka? Never heard of her.I’m using REST to talk to a service I’m proxying REST. And also logging stuff to Kafka
  36. 36. Vision #3: Kafka as a Service in a Mesh
  37. 37. 47 Like This: Proxy Proxy Proxy Proxy Proxy
  38. 38. Vision #4: Front Kafka
  39. 39. 52 Like This: P R O X Y P R O X Y
  40. 40. Lets Sum Things Up 53
  41. 41. - REST API point-to-point communication hits the wall - Service Mesh adds discovery and operationalibility - Kafka adds data, compatibility, immutable storage - You can have both. In many different tastes. - Coming soon to a proxy near you
  42. 42. Resources and Next Steps https://github.com/confluentinc/cp-demo https://www.confluent.io/download/ https://slackpass.io/confluentcommunity https://www.confluent.io/blog https://www.confluent.io/confluent-operator/ https://github.com/envoyproxy/envoy/issues/2852

×