CQRS (Command Query Responsibility Segregation) is an architectural pattern that's talked about much and understood less. The gist of CQRS is that commands in your architecture are made differently than queries. The implication of this is that your entire architecture becomes unidirectional. No more request response models from simple APIs! How the heck do you build something like that? Despite being originally popularized in the .NET world over a decade ago, CQRS is making a comeback as unidirectional UI frameworks like Flux and Redux gain in popularity. If you're going to have a reactive, unidirectional user interface, why not have the same patterns reflected in your entire architecture? Matt will go over the tooling, patterns, and practices he's used to build CQRS architectures in marketing, finance, and more since he hopped on the Node bandwagon in 2011! Find the video at http://www.nycnode.com/videos/matt-walters-how-to-cqrs-in-node-eventually-consistent-unidirectional-systems-with-microservices

1. Matt Walters
@mateodelnorte
hi@iammattwalters.com
How to CQRS in Node
Eventually consistent architectures that scale
and grow.

2. It’s real. It works!
Former TechStars Co. CTO. Now consultant.
Built two businesses’ platforms from scratch using
CQRS in Node. Both large, distributed systems.
CQRS made both more maintainable and extendable.
Marketing platform crunched the Twitter firehose in
realtime. 3 Engineers managed around 20 services.
- GoChime.com
Bond Exchange with over $1.75B in trades. 6 engineers
managed around 40 services.
- Electronifie.com
Open Source my tooling and frameworks.
Other companies using them too!
(and I’ll tell you how)
That’s me ^^ !!

3. When to CQRS
• Realtime, reactive systems
• When preferring small, modular services
• When aiming for learning and growth
• When aiming to grow or eventually split teams
• Want ability to scale different parts of your
system separately
When not to CQRS
• Standalone, static sites
• Standalone, simple CRUD applications

4. What is CQRS?

5. First, a primer from history.
Bertrand Meyer, regarding object interfaces:
“Every method should either be a command that
performs an action, or a query that returns data to the
caller, but not both. In other words, Asking a question
should not change the answer.”
Long before CQRS was CQS:
Command Query Separation.
this guy

6. Some Type
doSomething() : void
getSomeStuff() : Stuff
Either change stuff<——
Or get stuff<——
CQS
Command-Query
Separation
doAndGetStuff() : Stuff Never both!
——————

7. Command-Query
Responsibility Segregation
A system-wide architecture that states - externally
facing subsystems (apps and apis) send commands to
perform actions which update the system’s state and
request queries to determine system’s state.
*Basically CQS on a system-wide scale. Calls between
services should change stuff, or get stuff. Never both.
CQRS

8. Command-Query
Responsibility Segregation
CQRS also denotes that queries and command
processing are provided by different subsystems.
Queries are made against a data store. Commands are
sent to and processed by services.
CQRS One more thing!

9. What is CQRS?
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web
client
denormalizer
web-uiweb-uisvc
unidirectional flow
eventually consistent
Queries
Commands
the dance!
Events

10. How about a larger system?
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
Queries
Commands
unidirectional flow
eventually consistent
the dance!
Events

11. What’s that dance you’re doing?
denormalizer
dbdenormalizer
dbdenormalizer
db
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
the dance!
chain reaction of events which play out as a result of an
incoming command.
each service subscribes to the events they care about
choreography!
(not orchestration)
Events

12. commands tell services
when an actor wants an action
clients send commands to instruct a service to do work
commands are sent asynchronously; fire and forget
commands are present-tense, directives: order.create
web app order-svc
order.create
commands are sent directly to a single receiving service

13. events tell the world
when you’re done
services publish to inform other services of work / actions performed,
and state updated
services publish (broadcast) events to any services
that wish to subscribe
events past-tense, describe what happened: order.created
order-svc fulfillment-svc
order.createdorder.created
order.created
order.created

14. Two types of services
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
front end

15. denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
back end
Two types of services

16. web-uiweb-uiweb-app
web
client
denormalizer
web-uiweb-uisvc
front end
denormalizer
dbdenormalizer
dbdenormalizer
db
What’s different?
Let’s focus on:

17. web-uiweb-uiweb-app
web
client
web-uiweb-uisvc
front end (an app’s perspective)
denormalizer
dbdenormalizer
dbdenormalizer
db
What’s different?
Apps (and apis) still query a db to get the state of the system
Never directly modify the db they read from
Let’s focus on:

18. web-uiweb-uiweb-app
web
client
web-uiweb-uisvc
denormalizer
dbdenormalizer
dbdenormalizer
db
What’s different?
Instead, apps (and apis) send commands instructing services to
perform an action
Apps expect their read only representation of system state will
eventually be updated in the denormalizer db
Let’s focus on:
front end (an app’s perspective)
Commands are sent over a reliable transport (rabbitmq, kafka,
zeromq, etc) to ensure delivery and eventual consistency

19. Let’s pick a transport!

20. rabbitmq
messaging that just works
• direct send to queue
• fanout / topic routing
• highly available
• highly performant
• used in financial exchanges,
industrial applications and more
• open source
• free

21. rabbitmq
messaging that just works
brew install rabbitmq

22. framework!minimalist
Let’s pick a

23. servicebus
super simple messaging in node
• direct send
• pub / sub / fanout / topic -routing
• simple to set up
• highly performant
• used in financial exchanges,
online advertising and more
• open source
• free
• perfect for creating microservices!

24. servicebus
super simple messaging in node
npm install servicebus —save

25. web-uiweb-uiweb-app
web
client
denormalizer
web-uiweb-uisvc
front end
denormalizer
dbdenormalizer
dbdenormalizer
db
What’s different?
Let’s focus on:

26. web-uiweb-uiweb-app
web-ui
web-uisvc
front end commands
denormalizer
dbdenormalizer
dbdenormalizer
db
How’s this work?
Let’s focus on:

27. Sending commands
from the front end
// web-app, onButtonClick. instead of updating db.
const bus = require('servicebus').bus();
bus.send(‘order.create', {
order: {
userId: userId,
orderItems: items
}
});
web-uiweb-uiweb-app
web-ui
web-uisvc
// fire and forget.
command name
command itself
order.create command

28. Then what from the front end?
web-uiweb-uiweb-app
denormalizer
dbdenormalizer
dbdenormalizer
db
We wait.
• reactive / realtime:
• mongo oplog tailing (meteor)
• rethinkdb
• redis notifications
• couchdb
• graphQL
• polling
• non-realtime:
• product design: thanks! we’re processing your
order! check back later for updates!
queries!

29. We wait.
For the backend.
Choreography.
It’s eventually consistent!

30. denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
front end
Let’s focus on:

31. denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
back end
Let’s focus on:

32. back end
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
Let’s focus on:

33. back end (a service’s perspective)
web-uiweb-uiweb-app
svc
Commands
Events• Listen for commands and subscribe to events
• Performs business logic to process commands and events
• Update local state (optionally)
• Publish events to tell external services of updated state
Let’s focus on:

34. Sample service.
web-uiweb-uiweb-app
web-ui
web-uisvc
command name
command object
// order-svc index.js
const bus = require(‘./bus’);
const create = require(‘./lib/create’);
bus.listen(‘order.create', (event) => {
create(event, (err, order) => {
if (err) return event.handle.reject();
bus.publish(‘order.created’, order, () => {
event.handle.ack();
});
});
});
service publishes to the world when it’s done!
order.create command
order.created
event

35. Sample service.
web-uiweb-uiweb-app
web-ui
web-uisvc
listening for commands
performing business logic &
updating state// order-svc index.js
const bus = require(‘./bus’);
const create = require(‘./lib/create’);
bus.listen(‘order.create', (event) => {
create(event, (err, order) => {
if (err) return event.handle.reject();
bus.publish(‘order.created’, order, () => {
event.handle.ack();
});
});
});
order.create command
order.created
event
completing atomic transaction and allowing error handling

36. back end (a downstream service’s perspective)
web-uiweb-uiweb-app
svc
Commands
Events• Listen for commands and subscribe to events
• Performs business logic to process commands and events
• Update local state (optionally)
• Publish events to tell external services of updated state
svc-2
Events
Events
Let’s focus on:
Same thing!

37. Sample downstream service.
web-uiweb-uiweb-app
web-ui
web-uisvc
// fulfillment-svc index.js
const bus = require(‘./bus’);
const fulfill = require(‘./lib/fulfill’);
bus.subscribe(‘order.created', (event) => {
fulfill(event, (err, order) => {
if (err) return event.handle.reject();
bus.publish(‘order.fulfilled’, order, () => {
event.handle.ack();
});
});
});
order.created
order.fulfilled
subscribe for events instead of
listening for commands
no different, from any other service!

38. Multiple handlers?
svc
Commands
svc-2
Events
svc-2
Events

39. servicebus-register-handlers
convention based event handler definition for
distributed services using servicebus.
automatically registers
event & command
handlers saved as
modules in folder
initialize at startup

40. servicebus-register-handlers
const bus = require(‘./lib/bus'); // instantiate servicebus instance
const config = require('cconfig')();
const log = require('llog');
const registerHandlers = require('servicebus-register-handlers');
registerHandlers({
bus: bus,
handleError: function handleError (msg, err) {
log.error('error handling %s: %s. rejecting message w/ cid %s and correlationId %s.', msg.type,
err, msg.cid, this.correlationId);
log.error(err);
msg.handle.reject(function () {
throw err;
});
},
path: './lib/handlers',
queuePrefix: 'my-svc-name'
});
initialize at startup:
provide initialized bus
define your error handling
path to your handlers
prefix to differentiate similar queues

41. servicebus-register-handlers
const log = require("llog");
module.exports.ack = true;
module.exports.queueName = 'my-service-name-order';
module.exports.routingKey = "order.create";
module.exports.listen = function (event, cb) {
log.info(`handling listened event of type ${event.type} with routingKey $
{this.routingKey}`);
/*
do something with your event
*/
cb();
};
each handler is a file
no params marks success.
pass back error to retry or fail.
callback based transactions!
differentiate queues
for different services
specify which commands or
events to listen or subscribe to

42. servicebus-register-handlers
super simple messaging in node
npm install servicebus-register-handlers —save

43. What about the ‘work’ part?
const log = require("llog");
module.exports.ack = true;
module.exports.queueName = 'my-service-name-order';
module.exports.routingKey = "order.create";
module.exports.listen = function (event, cb) {
log.info(`handling listened event of type ${event.type}
with routingKey ${this.routingKey}`);
/*
do something with your event
*/
cb();
};
// order-svc index.js
const bus = require(‘./bus’);
const create = require(‘./lib/create’);
bus.listen(‘order.create', (event) => {
create(event, (err, order) => {
if (err) return event.handle.reject();
bus.publish(‘order.created’, order, () => {
event.handle.ack();
});
});
});
these parts

44. What about the ‘work’ part?
That’s up to you!
Need an audit trail? Targeting finance? Consider event sourcing.
*and my framework, ‘sourced’
Depending on your problem, the right choice could be
mongoose and mongodb, a graph database, an in-memory data
structure, or even flat files.
CQRS makes no assertions about what technology you should
use, and in fact frees you to make a different decision for each
particular problem.
and depends on the problem you’re solving

45. But wait! There’s more!
servicebus middleware!
middleware can inspect and modify incoming
and outgoing messages
// ./lib/bus.js required as single bus instance used anywhere in service
const config = require('cconfig')();
const servicebus = require('servicebus');
const retry = require('servicebus-retry');
const bus = servicebus.bus({
url: config.RABBITMQ_URL
});
bus.use(bus.package());
bus.use(bus.correlate());
bus.use(retry({
store: new retry.RedisStore({
host: config.REDIS.HOST,
port: config.REDIS.PORT
})
}));
module.exports = bus;
bus.use() middleware
into bus message
pipeline. middleware can
act on incoming and/or
outgoing messages

46. But wait! There’s more!
servicebus middleware!
middleware can inspect and modify incoming
and outgoing messages
// ./lib/bus.js required as single bus instance used anywhere in service
const config = require('cconfig')();
const servicebus = require('servicebus');
const retry = require('servicebus-retry');
const bus = servicebus.bus({
url: config.RABBITMQ_URL
});
bus.use(bus.package());
bus.use(bus.correlate());
bus.use(retry({
store: new retry.RedisStore({
host: config.REDIS.HOST,
port: config.REDIS.PORT
})
}));
module.exports = bus;
packages outgoing
message data and
adds useful type,
timestamp, and
other properties

47. But wait! There’s more!
servicebus middleware!
middleware can inspect and modify incoming
and outgoing messages
// ./lib/bus.js required as single bus instance used anywhere in service
const config = require('cconfig')();
const servicebus = require('servicebus');
const retry = require('servicebus-retry');
const bus = servicebus.bus({
url: config.RABBITMQ_URL
});
bus.use(bus.package());
bus.use(bus.correlate());
bus.use(retry({
store: new retry.RedisStore({
host: config.REDIS.HOST,
port: config.REDIS.PORT
})
}));
module.exports = bus;
adds a correlationId
for tracing related
commands and events
through your system

48. But wait! There’s more!
servicebus middleware!
middleware can inspect and modify incoming
and outgoing messages
// ./lib/bus.js required as single bus instance used anywhere in service
const config = require('cconfig')();
const servicebus = require('servicebus');
const retry = require('servicebus-retry');
const bus = servicebus.bus({
url: config.RABBITMQ_URL
});
bus.use(bus.package());
bus.use(bus.correlate());
bus.use(retry({
store: new retry.RedisStore({
host: config.REDIS.HOST,
port: config.REDIS.PORT
})
}));
module.exports = bus;
now, every failed
message will retry
3 times if errors occur.
after that, the message
will automatically be
put on an error queue
for human inspection!

49. And more!
distributed tracing middleware!
var trace = require('servicebus-trace');
bus.use(trace({
serviceName: 'my-service-name',
store: new trace.RedisStore({
host: config.REDIS_HOST || 'localhost',
port: config.REDIS_PORT || 6379
})
}));

50. denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
back end
Recapping Back End Services

51. back end services
web-uiweb-uiweb-app
svc
Commands
Events• Listen for commands and subscribe to events
• Performs business logic to process commands and events
• Update local state (optionally)
• Publish events to tell external services of updated state
Recapping:

52. denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
back end
Recapping Back End Services
wat wat watwat

53. Wat!
is
a
denormalizer?

54. back end
What’s a denormalizer?
front end
• Just another back end service
• Has one job to do
• Subscribe to all events that the UI cares about
• Persist events in a format most efficient for the UI to view
• Completes the eventually consistent, unidirectional flow
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web
client
denormalizer
Events
svc

55. denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web
client
denormalizer
back end
What’s a denormalizer?
Events
front end
svc

56. Recapping the big picture.
denormalizer
dbdenormalizer
dbdenormalizer
db
web-uiweb-uiweb-app
web-uiweb-uiweb-api
web
client
mobile
client
denormalizer
svc-3
svc-2
web-uiweb-uisvc-1
Queries
Commands
unidirectional flow
eventually consistent
the dance!
Events

57. CQRS? That’s a myth!

58. Thanks!
We’ll sneak preview even more tooling for
easily building these systems in a future talk!

59. Matt Walters
github & twitter: @mateodelnorte
email: hi@iammattwalters.com
website: iammattwalters.com
rabbitmq.com
npmjs.com/package/servicebus
How to CQRS in Node
npmjs.com/package/servicebus-register-handlers
npmjs.com/package/servicebus-retry
npmjs.com/package/servicebus-trace