This is the talk that the Kenny Bastani and I gave at JavaOne 2016.
When you’re building JVM applications in a microservice architecture, managing state becomes a distributed systems problem. Instead of being able to manage state as transactions inside the boundaries of a single monolithic application, a microservice must be able to manage consistency by using transactions that are distributed across a network of many different applications and databases. This session explores the problems of data consistency and high availability in JVM-based microservices and how to use event sourcing to solve these problems.
5. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS
11. @crichardson
Event sourcing
For each domain object (i.e. DDD aggregate):
Identify (state changing) domain events, e.g. use Event
Storming
Define Event classes
For example, Order events: OrderCreated, OrderCancelled,
OrderApproved, OrderRejected, OrderShipped
12. @crichardson
Persists events NOT current state
Event table
Entity type
Event
id
Entity
id
Event
data
Order 902101 …OrderApproved
Order 903101 …OrderShipped
Event
type
Order 901101 …OrderCreated
13. @crichardson
Replay events to recreate
state
Order
state
OrderCreated(…)
OrderAccepted(…)
OrderShipped(…)
Events
Periodically snapshot to avoid loading all events
14. @crichardson
Benefits of event sourcing
Reifies state changes:
Built in, reliable audit log
temporal queries
Preserved history More easily implement future requirements
Eliminates O/R mapping problem (mostly)
Reliable event publishing: publishes events needed by
predictive analytics etc, user notifications,…
15. @crichardson
Drawbacks of event sourcing
Requires application rewrite
Weird and unfamiliar style of programming
Events live forever carefully evolve schema
Querying the event store can be challenging
Current state is no longer directly available
Often need to maintain views for efficiency
17. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS
21. @crichardson
ACID transactions cannot be
used
BEGIN TRANSACTION
…
SELECT ORDER_TOTAL
FROM ORDERS WHERE CUSTOMER_ID = ?
…
SELECT CREDIT_LIMIT
FROM CUSTOMERS WHERE CUSTOMER_ID = ?
…
INSERT INTO ORDERS …
…
COMMIT TRANSACTION
Private to the
Order Service
Private to the
Customer Service
Requires 2PC
22. @crichardson
2PC is not a viable option
Guarantees consistency
BUT
2PC is best avoided
Not supported by many NoSQL databases etc.
CAP theorem 2PC impacts availability
….
23. @crichardson
Queries can’t use joins
SELECT *
FROM CUSTOMER c, ORDER o
WHERE
c.id = o.ID
AND o.ORDER_TOTAL > 100000
AND o.STATE = 'SHIPPED'
AND c.CREATION_DATE > ?
Customer
Service
Order Service
24. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS
26. @crichardson
Order Management
Order
id : 4567
total: 343
state = CREATED
Customer Management
Customer
creditLimit : 12000
creditReservations: {}
Customer
creditLimit : 12000
creditReservations: { 4567 -> 343}
Order
id : 4567
total: 343
state = APPROVED
Eventually consistent credit checking
Message Bus
createOrder()
Publishes:
Subscribes to:
Subscribes to:
publishes:
OrderCreatedEvent
CreditReservedEvent
OrderCreatedEvent CreditReservedEvent
31. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS
33. @crichardson
Event Store
Application architecture
Order 123 Customer 456
OrderCreated
OrderApproved
…
CustomerCreated
CustomerCreditReserved
…
CreateOrder
UpdateOrder
GetOrder
Subscribe
Order
Service
CreateCustomer
UpdateCustomer
GetCustomer
Subscribe
Customer
Service
34. @crichardson
Request handling in an event sourced application
HTTP
Handler
Event
Store
pastEvents = findEvents(entityId)
Order
new()
applyEvents(pastEvents)
newEvents = processCmd(someCmd)
saveEvents(newEvents) (optimistic locking)
Order Service
applyEvents(newEvents)
35. @crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
publish(event)
publish(event)
Customer
update()
Customer Service
36. @crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
publish(event)
publish(event)
CQRS View
update()
Service Xyz
send notifications
…
37. Event store = database + message
broker
Hybrid database and
message broker
Implementations:
Home grown/DIY
geteventstore.com by
Greg Young
http://eventuate.io
(mine)
Event Store
Save
aggregate
events
Get
aggregate
events
Subscribe
to events
38. @crichardson
Agenda
Overview of event sourcing
The problem with microservices, transactions and queries
Using events to maintain consistency
Event sourcing in a microservice architecture
Implementing queries with CQRS
39. @crichardson
Find recent, valuable
customers
SELECT *
FROM CUSTOMER c, ORDER o
WHERE
c.id = o.ID
AND o.ORDER_TOTAL > 100000
AND o.STATE = 'SHIPPED'
AND c.CREATION_DATE > ?
Customer
Service
Order Service
What if event
sourcing is
used?…. is no longer easy
43. @crichardson
Query side design
Event Store
Updater
View Updater
Service
Events
Reader
HTTP GET
Request
View Query
Service
View
Store
e.g.
MongoDB
ElasticSearch
Neo4J
update query
49. @crichardson
Summary
Microservice architecture functionally decomposes an
application into services
Transactions and queries resist decomposition
Use an event-driven architecture based on event sourcing to
maintain data consistency
Implement queries using CQRS