O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
@rafabene e @roanbrasil - https://bit.ly/sagakafka 1
Saga Pattern and Event
Sourcing with Kafka
https://bit.ly/sagakafkaLi...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
rafael.benevides@gmail.com
@rafabene
apiVersion: oracle/v1
kind: Princi...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
roanbrasil@gmail.com
@roanbrasil
apiVersion: astek-canada/v1
kind: Java...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Before we get started…
● I don't use microservices and love my monolith...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Monolithic architecture
Order
Module
Booking
Module
Payment
Module
Web
...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Moving to Microservices
Database
Browser /
Client
Order
Module
Booking
...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Moving to Microservices
Database
Browser /
Client
Order
Service
Booking...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Microservices Architecture (synchronous calls)
Database
Browser /
Clien...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
No ACID transaction that span services
Database
Browser /
Client
Order
...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Exceptions: No room available (synchronous calls)
Database
Browser /
Cl...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Exceptions: Payment Denied (synchronous calls)
Database
Browser /
Clien...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Death Star Architectures
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Can I use 2PC?
Prepare phase
Database
Browser /
Client
Order
Service
Bo...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Can I use 2PC?
Prepare phase
Database
Browser /
Client
Order
Service
Bo...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Can I use 2PC?
Commit phase
Database
Browser /
Client
Order
Service
Boo...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Can I use 2PC?
Commit phase
Database
Browser /
Client
Order
Service
Boo...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Saga: A solution from 1987
https://www.cs.cornell.edu/andru/cs711/2002f...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
How does the saga work?
Order
Service
Order
createOrder()
Order{
state ...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
How does the rollback work?
Use compensating transactions to "undo" the...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Rollback: Payment Denied
Order
Service
Order
createOrder()
Order{
state...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Rollback exception: Payment service unavailable
Order
Service
Order
cre...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Event Sourcing
(An express explanation)
@rafabene e @roanbrasil - https://bit.ly/sagakafka
What is event sourcing?
Architectural style where you:
● Don't save the...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Creating events to change to state
BankAccountCreated
Id: 123
Owner: Ra...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Creating events to change to state
MoneyWithdraw
account: 123
value: 60...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Command Sourcing
(An express explanation)
@rafabene e @roanbrasil - https://bit.ly/sagakafka
What is command sourcing?
Architectural style where you:
● Persist comm...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Saga Patterns
● Choreography-based sagas - services collaborate by exch...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Saga Pattern:
Choreography-based
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Order
Service
Event Store
createOrder()
Publish
OrderCreatedEvent
Booki...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Order
Service
Event Store
createOrder()
Publish
OrderCreatedEvent
Booki...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Order
Service
Event Store
createOrder()
Publish
OrderCreatedEvent
Booki...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Advantages and disadvantages of Choreography
● Simple to implement
● Ti...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Event Command Transformation Pattern
Order
Service
NoRoomEvent
BookConf...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Event Command Transformation Pattern
Order
Service
Booking
ServiceOrder...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Order
Service
Broker
createOrder()
Publish
BookingCommand
Booking
Servi...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Saga Pattern:
Orchestration-based
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Order
Service
Broker
createOrder()
Publish
OrderCreatedEvent
Consume
Or...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Order
Service
Broker
createOrder()
Publish
OrderCreatedEvent
Consume
Or...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Advantages and disadvantages of Orchestration
● Centralized business lo...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
A note about
Aggregator vs Chain
Patterns
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Choreographed
Aggregator
(Parallel)
Chain
(Serial) Order
Service
Bookin...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Orchestrated
Aggregator (Parallel)
Chain (Serial)
Order
Service
Booking...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Aggregator vs Chain Patterns
● Which pattern do you think it is easier
...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Event sourcing with Kafka
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Creating an Orchestrator
using Kafka Streams
@rafabene e @roanbrasil - https://bit.ly/sagakafka
API to Consume, process and publish to Kafka Topics
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Asynchronous chained orchestrated saga
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Demo use cases
● Book a room
● Payment offline
● Payment denied
● No ro...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Demo
rafabene/saga-demo
@rafabene e @roanbrasil - https://bit.ly/sagakafka
Run them on Linux
VirtualPhysical Private Public
Oracle Linux
Node Node...
@rafabene e @roanbrasil - https://bit.ly/sagakafka
@rafabene e @roanbrasil - https://bit.ly/sagakafka@rafabene e @roanbrasil
@RAFABENE
@ROANBRASIL
Próximos SlideShares
Carregando em…5
×

1

Compartilhar

Baixar para ler offline

Saga pattern and event sourcing with kafka

Baixar para ler offline

Trilha NodeJS com meu amigo Rafael Benevides

Saga pattern and event sourcing with kafka

  1. 1. @rafabene e @roanbrasil - https://bit.ly/sagakafka 1 Saga Pattern and Event Sourcing with Kafka https://bit.ly/sagakafkaLink @rafabene rafael.benevides@oracle.com @roanbrasil rmonteiro@astek-canada.ca
  2. 2. @rafabene e @roanbrasil - https://bit.ly/sagakafka rafael.benevides@gmail.com @rafabene apiVersion: oracle/v1 kind: PrincipalSolutionArchitect metadata: name: Rafael Benevides namespace: Oracle Linux team annotations: apache/contributor: Apache DeltaSpike PMC labels: developer: Java, NodeJS hobby: 4x4, drones spec: replicas: 1 containers: image: benevides/rafael:latest Rafael Benevides
  3. 3. @rafabene e @roanbrasil - https://bit.ly/sagakafka roanbrasil@gmail.com @roanbrasil apiVersion: astek-canada/v1 kind: JavaDeveloper metadata: name: Roan Brasil Monteiro namespace: Java Developer team labels: developer: Java, NodeJS hobby: guitar, video game, snorkeling, soccer spec: replicas: 1 containers: image: brasilmonteiro/roan:latest Roan Brasil Monteiro
  4. 4. @rafabene e @roanbrasil - https://bit.ly/sagakafka Before we get started… ● I don't use microservices and love my monolith. ● I've started using microservices. ● I've heard about Event Sourcing and Sagas before. ● I do microservices using Event Sourcing at work. ● I've implemented Saga Patterns. Who are you?
  5. 5. @rafabene e @roanbrasil - https://bit.ly/sagakafka
  6. 6. @rafabene e @roanbrasil - https://bit.ly/sagakafka Monolithic architecture Order Module Booking Module Payment Module Web UI Database Browser / Client BEWARE: This is NOT an anti-pattern!
  7. 7. @rafabene e @roanbrasil - https://bit.ly/sagakafka Moving to Microservices Database Browser / Client Order Module Booking Module Payment Module Web UI
  8. 8. @rafabene e @roanbrasil - https://bit.ly/sagakafka Moving to Microservices Database Browser / Client Order Service Booking Service Payment Service Web UI Database Database
  9. 9. @rafabene e @roanbrasil - https://bit.ly/sagakafka Microservices Architecture (synchronous calls) Database Browser / Client Order Service Booking Service Payment Service Web UI Database Database 1) New Order 2) Room Reservation 3) ProcessPayment REST Calls
  10. 10. @rafabene e @roanbrasil - https://bit.ly/sagakafka No ACID transaction that span services Database Browser / Client Order Service Booking Service Payment Service Web UI Database Database 1) New Order 2) Room Reservation 3) ProcessPayment T1 T2 T3 REST Calls
  11. 11. @rafabene e @roanbrasil - https://bit.ly/sagakafka Exceptions: No room available (synchronous calls) Database Browser / Client Order Service Booking Service Web UI Database1) New Order 2) Room Reservation 3) Exception happened4) Cancel order REST Calls 1:1 services involved That was easy!
  12. 12. @rafabene e @roanbrasil - https://bit.ly/sagakafka Exceptions: Payment Denied (synchronous calls) Database Browser / Client Order Service Booking Service Payment Service Web UI Database Database 1) New Order 2) Room Reservation 3) ProcessPayment 6) Cancel Booking???6) Cancel Booking??? REST Calls 4) Exception happened 5) Cancel order ● What if I have 10 microservices to rollback? ● Can you spot the complexity?
  13. 13. @rafabene e @roanbrasil - https://bit.ly/sagakafka Death Star Architectures
  14. 14. @rafabene e @roanbrasil - https://bit.ly/sagakafka Can I use 2PC? Prepare phase Database Browser / Client Order Service Booking Service Payment Service Web UI Database Database 1) New Order 2) Room Reservation 3) ProcessPayment Prepare Prepare Prepare
  15. 15. @rafabene e @roanbrasil - https://bit.ly/sagakafka Can I use 2PC? Prepare phase Database Browser / Client Order Service Booking Service Payment Service Web UI Database Database 1) New Order 2) Room Reservation 3) ProcessPayment Prepared Prepared Prepared
  16. 16. @rafabene e @roanbrasil - https://bit.ly/sagakafka Can I use 2PC? Commit phase Database Browser / Client Order Service Booking Service Payment Service Web UI Database Database 1) New Order 2) Room Reservation 3) ProcessPayment Commit/ Abort Commit/ Abort Commit/ Abort
  17. 17. @rafabene e @roanbrasil - https://bit.ly/sagakafka Can I use 2PC? Commit phase Database Browser / Client Order Service Booking Service Payment Service Web UI Database Database 1) New Order 2) Room Reservation 3) ProcessPayment Done Done Done Problems of 2 Phase commit ● Transaction coordinator is the SPF. ● Others have to wait until the slowest resource finish its confirmation. ○ Ex. Credit card payment can take up to 30 seconds to reply. ○ Databases are locked during that time. ○ Can cause deadlocks. ● Reduced throughput due to locks. ● Not supported by many NoSQL databases.
  18. 18. @rafabene e @roanbrasil - https://bit.ly/sagakafka Saga: A solution from 1987 https://www.cs.cornell.edu/andru/cs711/2002fa/reading/sagas.pdf
  19. 19. @rafabene e @roanbrasil - https://bit.ly/sagakafka How does the saga work? Order Service Order createOrder() Order{ state = PENDING } Order Service Order Order{ state = APPROVED } Payment Service Account PaymentInfo{ state = APPROVED }
  20. 20. @rafabene e @roanbrasil - https://bit.ly/sagakafka How does the rollback work? Use compensating transactions to "undo" the business logic. Order createOrder() Booking reserveRoom() Payment withdraw() Order cancelOrder() No rooms available Booking cancel Reservation() T1 T2 T3 C1 C2 T1 T2 C1 T3 and C2 were not called
  21. 21. @rafabene e @roanbrasil - https://bit.ly/sagakafka Rollback: Payment Denied Order Service Order createOrder() Order{ state = PENDING } Order Service Order Order{ state = CANCELED } Payment Service Account PaymentInfo{ state = DENIED }
  22. 22. @rafabene e @roanbrasil - https://bit.ly/sagakafka Rollback exception: Payment service unavailable Order Service Order createOrder() Order{ state = PENDING } Order Service Order Order{ state = PENDING } Payment Service Account REST Call Client and Server must be always available Order will be PENDING forever We need to go asynchronous
  23. 23. @rafabene e @roanbrasil - https://bit.ly/sagakafka Event Sourcing (An express explanation)
  24. 24. @rafabene e @roanbrasil - https://bit.ly/sagakafka What is event sourcing? Architectural style where you: ● Don't save the current state of objects ● Save the events that lead to the current state (ORDER IS IMPORTANT) ● Events happened in the past ● Example: AccountDeposit accountNumber: 123456-700 value: 100.50
  25. 25. @rafabene e @roanbrasil - https://bit.ly/sagakafka Creating events to change to state BankAccountCreated Id: 123 Owner: Rafael BankAccount Events State BankAccount Id: 123 Owner: Rafael Balance: 0 MoneyDeposited account: 123 value: 100 BankAccount Id: 123 Owner: Rafael Balance: 100
  26. 26. @rafabene e @roanbrasil - https://bit.ly/sagakafka Creating events to change to state MoneyWithdraw account: 123 value: 60 Events State BankAccount Id: 123 Owner: Rafael Balance: 100 BankAccount Id: 123 Owner: Rafael Balance: 40 MoneyWithdraw account: 123 value: 60 BankAccount Id: 123 Owner: Rafael Balance: 40 Event not possible to be handled
  27. 27. @rafabene e @roanbrasil - https://bit.ly/sagakafka Command Sourcing (An express explanation)
  28. 28. @rafabene e @roanbrasil - https://bit.ly/sagakafka What is command sourcing? Architectural style where you: ● Persist commands ● Says what command/intention should be performed ● Wait for replies ● Example: ExecuteAccountWithdraw accountNumber: 123456-700 value: 100.50 InsuficientBalanceReply accountNumber: 123456-700 requestedValue: 100.50
  29. 29. @rafabene e @roanbrasil - https://bit.ly/sagakafka Saga Patterns ● Choreography-based sagas - services collaborate by exchanging domain events ● Orchestration-based sagas - a centralized coordinator sending command messages to participants, which respond with reply messages https://eventuate.io/post/eventuate/2020/02/24/why-eventuate.html
  30. 30. @rafabene e @roanbrasil - https://bit.ly/sagakafka Saga Pattern: Choreography-based
  31. 31. @rafabene e @roanbrasil - https://bit.ly/sagakafka Order Service Event Store createOrder() Publish OrderCreatedEvent Booking Service Consume OrderCreatedEvent Publish BookCreatedEvent Payment Service Consume BookCreatedEvent Publish PaymentSuccess Consume BookConfirmedEvent Choreography Consume PaymentSuccess Publish BookConfirmedEvent
  32. 32. @rafabene e @roanbrasil - https://bit.ly/sagakafka Order Service Event Store createOrder() Publish OrderCreatedEvent Booking Service Consume OrderCreatedEvent Publish NoRoomEvent Payment Service Consume NoRoomEvent Choreography: No rooms available Compensating transaction
  33. 33. @rafabene e @roanbrasil - https://bit.ly/sagakafka Order Service Event Store createOrder() Publish OrderCreatedEvent Booking Service Consume OrderCreatedEvent Publish BookCreatedEvent Payment Service Consume BookCreatedEvent Publish PaymentDenied Consume BookCanceledEvent Choreography: Payment denied Compensating transactions Consume PaymentDenied Publish BookCanceledEvent
  34. 34. @rafabene e @roanbrasil - https://bit.ly/sagakafka Advantages and disadvantages of Choreography ● Simple to implement ● Tight coupling - highly dependent upon each other ● The decision logic is distributed - difficulty to maintain and manage the overall process ● Hard to debug
  35. 35. @rafabene e @roanbrasil - https://bit.ly/sagakafka Event Command Transformation Pattern Order Service NoRoomEvent BookConfirmedEvent ... Booking has to know all events that triggers a Order action BookCanceledEvent
  36. 36. @rafabene e @roanbrasil - https://bit.ly/sagakafka Event Command Transformation Pattern Order Service Booking ServiceOrderCreatedEvent BookCommand Event Something has happened in the past Command Something has to happen in the future Event Command Transformation Where we do the coupling
  37. 37. @rafabene e @roanbrasil - https://bit.ly/sagakafka Order Service Broker createOrder() Publish BookingCommand Booking Service Consume BookingCommand Publish PaymentCommand Payment Service Consume PaymentCommand Publish BookCommand Consume OrderCommand Event Command Transformation Pattern
  38. 38. @rafabene e @roanbrasil - https://bit.ly/sagakafka Saga Pattern: Orchestration-based
  39. 39. @rafabene e @roanbrasil - https://bit.ly/sagakafka Order Service Broker createOrder() Publish OrderCreatedEvent Consume OrderConfirmation Command Orchestration Orchestrator Consume OrderCreatedEvent Produce ReservationCommand Booking Service Consume Reservation Command Payment Service Publish ReservationResult Consume Payment Command Publish PaymentResult Consume ReservationResult Produce PaymentCommand Consume PaymentResult Produce BookingConfirmation Command OrderConfirmation Command Consume Booking Confirmation Command
  40. 40. @rafabene e @roanbrasil - https://bit.ly/sagakafka Order Service Broker createOrder() Publish OrderCreatedEvent Consume OrderCancelation Command Orchestration: Payment Denied Orchestrator Consume OrderCreatedEvent Produce ReservationCommand Booking Service Consume Reservation Command Payment Service Publish ReservationResult Consume Payment Command Publish PaymentDenied Consume ReservationResult Produce PaymentCommand Consume PaymentDennied Produce BookingCancelationC omand and OrderCancelation Command Consume Booking Cancelation Command
  41. 41. @rafabene e @roanbrasil - https://bit.ly/sagakafka Advantages and disadvantages of Orchestration ● Centralized business logic - Easy to understand and maintain. ● Loose coupling as each service consumes only their commands ● Orchestrator is the SPF ● Needs Event -> Command transformation
  42. 42. @rafabene e @roanbrasil - https://bit.ly/sagakafka A note about Aggregator vs Chain Patterns
  43. 43. @rafabene e @roanbrasil - https://bit.ly/sagakafka Choreographed Aggregator (Parallel) Chain (Serial) Order Service Booking Service Payment Service Order Service Booking Service Payment Service Order triggers compensation Each service triggers compensation from previous transaction O rder holds the logic for the use case
  44. 44. @rafabene e @roanbrasil - https://bit.ly/sagakafka Orchestrated Aggregator (Parallel) Chain (Serial) Order Service Booking Service Payment Service Order Service Booking Service Payment Service Orchestrator Orchestrator Orchestrator needs to coordinate states before calling proper compensation transactions Orchestrator flow is easier to handle and implement
  45. 45. @rafabene e @roanbrasil - https://bit.ly/sagakafka Aggregator vs Chain Patterns ● Which pattern do you think it is easier to implement and handle States ● Tip:
  46. 46. @rafabene e @roanbrasil - https://bit.ly/sagakafka Event sourcing with Kafka
  47. 47. @rafabene e @roanbrasil - https://bit.ly/sagakafka Creating an Orchestrator using Kafka Streams
  48. 48. @rafabene e @roanbrasil - https://bit.ly/sagakafka API to Consume, process and publish to Kafka Topics
  49. 49. @rafabene e @roanbrasil - https://bit.ly/sagakafka Asynchronous chained orchestrated saga
  50. 50. @rafabene e @roanbrasil - https://bit.ly/sagakafka Demo use cases ● Book a room ● Payment offline ● Payment denied ● No rooms available ● Orchestrator offline?
  51. 51. @rafabene e @roanbrasil - https://bit.ly/sagakafka Demo rafabene/saga-demo
  52. 52. @rafabene e @roanbrasil - https://bit.ly/sagakafka Run them on Linux VirtualPhysical Private Public Oracle Linux Node NodeNode Master API ServerDev Ops SCM (Git/Svn) CI/CD Automation Controllers - Scheduler - Replication - Services - Builds - Routes - Deployment Kubernetes Ingress Gateway
  53. 53. @rafabene e @roanbrasil - https://bit.ly/sagakafka
  54. 54. @rafabene e @roanbrasil - https://bit.ly/sagakafka@rafabene e @roanbrasil @RAFABENE @ROANBRASIL
  • niksrn2011

    Sep. 1, 2020

Trilha NodeJS com meu amigo Rafael Benevides

Vistos

Vistos totais

342

No Slideshare

0

De incorporações

0

Número de incorporações

2

Ações

Baixados

5

Compartilhados

0

Comentários

0

Curtir

1

×