Slides from my presentation at O'Reilly Software Architecture 2018, co-presented with Allard Buijze. Much hyped architectural pattern CQRS is getting a lot of attention, but it does actually deliver on its promises of managing complexity and scalability when used with the right abstractions. Casumo, a Malta-based online casino, adopted the principles of CQRS based on these promises. As the company scaled to hundreds of employees and over a hundred services, these promises were put to the challenge. Allard Buijze and Nakul Mishra discuss the challenges Casumo faced while scaling its system to millions of financial transactions per day and applying event sourcing with billions of events to keep up with the ever-changing demands of the gaming industry.

1. allardbz nklmish
Scaling CQRS in Theory,
Practice and Reality
Allard Buijze
Founder & CTO, AxonIQ
allard@axoniq.io
allardbz
Nakul Mishra
Sr. Engineer & Architect
nklmish@protonmail.com
nklmish

2. allardbz nklmish

3. allardbz nklmish

4. allardbz nklmish

5. allardbz nklmish
Layered architecture
User Interface
Service Layer
Data Access Layer
DomainModel

6. allardbz nklmish
Service
Service
Service

7. allardbz nklmish
‘Normal’ SQL QUERY
22 JOINS 6 SUBQUERIES

8. allardbz nklmish
Layered architecture
Ponder and deliberate before you make a move. —Sun Tzu
User Interface
Service Layer
Data Access Layer
DomainModel
Method invocation Cache
Worker pools
Web
Cache
Session replication
Distributed 2nd level cache Query Cache

9. allardbz nklmish
Source: http://www.sabisabi.com/images/DungBeetle-on-dung.JPG

10. allardbz nklmish
Microservices systems
• Splittingup systemsinto smaller,
simplercomponents
• Agility
• Scalability

11. allardbz nklmish
CQRS

12. allardbz nklmish
Command Query Responsibility
Segregation
Command
model
Projections
Client
Events
T: 1 mln / s
Resp: < 10 ms
T: Thr. 20 / s
Resp: < 100 ms
T: 10 mln / s
Resp. < 100 ms
T: 1 / s
Resp. < 10 ms

13. allardbz nklmish

14. allardbz nklmish
Are you tall enough?
Source: martinfowler.com/bliki/MicroservicePrerequisites.html

15. allardbz nklmish
“Noun Driven Design”

16. allardbz nklmish
“Entity Services”
è

17. allardbz nklmish
Monoliths
St Breock Downs Monolith - www.cornwalls.co.uk

18. allardbz nklmish
$

19. allardbz nklmish
Location transparency
A Component should not be aware, nor make any
assumptions, of the location of Components it
interacts with
A component should neither be aware of nor make any
assumptions about the location of components it interacts with.
Location transparency starts with good API design
(but doesn’tend there)

20. allardbz nklmish
Microservices Messaging
Commands Events Queries
Route to single handler
Use consistent hashing
Provide result
Distribute to all logical handlers
Consumers express ordering req’s
No results
Route with load balancing
Sometimes scatter/gather
Provide result
"Event" and “Message" is not the same thing

21. allardbz nklmish

22. allardbz nklmish

23. allardbz nklmish
ComponentB
ComponentAWrites / event sourcing
consume event-A
ComponentC
Projection
consume event - A
Read / replay
Built with Axon Framework
publish event - C
BigData Platform
Readshistoricalevents
consumes new events
EventStore
(Percona)

24. allardbz nklmish
Casumo
• Event storewith >30 billionevents
• Hundreds of millions of events, every day
• Every event is EQUALLY important
“I was thinkingI might win 50 poundsbut when it went all the way
to the jackpotI was shocked.” - Mega Fortune£2,700,000 jackpot
won on the 3rd spin.

25. allardbz nklmish

26. allardbz nklmish

27. allardbz nklmish
Event Store operations
• Append
• Validate‘sequence’

28. allardbz nklmish
Event Store operations
• Full sequentialread

29. allardbz nklmish
Event Store operations
• Read aggregate’sevents

30. allardbz nklmish
Serialized form
• Keep it small
• Use aliases & abbreviations
• Carefully select serializer

31. allardbz nklmish
Partitioning and archiving

32. allardbz nklmish
0: Wallet created
€ 7 in wallet @ 3
2: € 1 wagered
3: € 2 wagered
1: € 10 deposited
4: € 1 wagered
5: € 2000 won
6: € 1000 withdrawn
7: € 10 wagered
8: € 1M jackpot won!!
€ 996 in wallet @ 7
Loading aggregates - snapshots

33. allardbz nklmish
Replays
Event Handler
TRUNCATE TABLE xyz;Projection

34. allardbz nklmish
Partial replays
Event Handler
TRUNCATE TABLE xyz;
“Reasonable” timeframe
Idempotent event
handling
Perform selective,
ad-hoc, replays
Projection

35. allardbz nklmish

36. allardbz nklmish
Exchange
Queue
Queue
Event Store

37. allardbz nklmish
Exchange
Queue
Queue
Event Store
Queue

38. allardbz nklmish

39. allardbz nklmish
Evolutionary microservices
Commands Events Queries

40. allardbz nklmish

41. allardbz nklmish

42. allardbz nklmish

43. allardbz nklmish
Unmanageable mess
Bet Placed
Bet Accepted
Bet Won
Bet Lost
Bet Reverted
As a Wallet
module I want to
know when to
withdraw money
from the wallet

44. allardbz nklmish
Communication = Contract

45. allardbz nklmish
Bounded context
Explicitlydefinethe context within which a model applies.Explicitly
set boundaries in terms of team organization,usage within specific
parts of the application, and physical manifestationssuch as code
bases and databaseschemas. Keep the model strictly consistentwithin
these bounds,but don’t be distractedor confused by issues outside.

46. allardbz nklmish
Within a context, share ‘everything’

47. allardbz nklmish
Between contexts, share ‘consciously’As a Wallet
module I want to
know when to
withdraw money
from the wallet
Bet Placed
Bet Accepted
Bet Won
Bet Lost
Bet Reverted
Bet placed
à Claim Funds

48. allardbz nklmish
Lessons learned

49. allardbz nklmish

50. allardbz nklmish

51. allardbz nklmish

52. allardbz nklmish

53. allardbz nklmish
Thank you!