In the Internet of things, data and commands between things and servers are sent as streams of events, which are often aggregated and processed to provide up to date information to end users.  Because of this, CQRS and Event Sourcing patterns are a natural fit for IoT applications.  In this presentation we provide an overview of these patterns, how they apply to IoT applications and their benefits. A prototype application of Event Sourcing is then demonstrated using the Sense Tecnic FRED platform based on Node-RED - a data flow programming tool for wiring up the internet of things

1. Event Sourcing for the IoT
Mike Blackstock
Sense Tecnic Systems, Inc.

2. Overview
• What is the IoT today?
• CRUD to Event Sourcing for the IoT
• Role of Node-RED in IoT systems
• Demonstrate Node-RED and FRED service for IoT app development

3. The IoT Today - Fog Computing
• Sense the world
• Track assets in a supply chain
• Soil moisture levels in fields
• Data from factory floor
• Actuate to change it
• Control lighting
• HVAC system control
• Control Irrigation
• The number of devices far exceeds
users….
• Cloud to Fog
• data, compute, storage and apps
distributed across devices, gateways
and cloudhttps://erpinnews.com/fog-computing-vs-edge-computing

4. IoT Application Requirements
• Present real world data to end users for decision making
• Add intelligence, user control, system control
• Variety of protocols, standards & formats
• HTTP, MQTT, Websockets, CoAP, OPC, Modbus, Bluetooth, WiFi, LoRaWAN…
• Integration with a variety of services
• Enterprise systems, alerting systems, virtual ‘sensors’
• “Real time”
• Send an alert when a tank level is too high
• Turn on irrigation system when moisture too low
• Send SMS when taxi is about to arrive
• Access to historical data
• Time series charts in visualizations
• Historical analysis, recognizing patterns, learning
• Storage and compute in gateways and devices (fog)
• Reduce latency, storage, data rates, and work disconnected

5. Traditional CRUD Applications
• IoT apps often start by leveraging traditional
frameworks
• Model of data where we create records,
update them, delete them
• Model consists of users, things, information
about things
• Things and users communicate with the system
using a REST API for example.
Update
Presentation
Data Storage
Business Logic
Data Access
Query

6. Challenges
• Scale
• many things and high data rates
• Valuable historical data (events) may be lost
• An update from a user or a thing may trigger many operations
• checked for alert condition
• notification to a user
• displayed in a dashboard
• processed and sent to an enterprise system
• trigger an actuator
• count for a subscription
• Integrating things & services

7. Adding things
• Things could communicate via API
• New controllers or presentation
servers
• Requires real time input and output --
Websockets?
• Different protocols required
• E.g. how to support MQTT
• security and authentication schemes
• Historical data
• time series data stores.
• Business logic to process data may
be complex, not easy to scale up.
State Storage
Business Logic
State Data
Access
Query
Users
Update
API
Time Series
Storage
Time Series
Access
Presentation
Things

8. CQRS: Increase Scalability
• Command Query Responsibility Segregation
• different logic and data models used to
update/write vs. query/read side
• Split commands from query model to scale
independently.
• E.g. mobile app and web pages rendered
using query model
• User interaction updates command model
in write storage
• Changes communicated to read storage
model either by the database, or
communication between command and
query processes
Presentation
Domain
Logic
Write
Storage
Generate
Views
Read
Storage
Query
Generation
Commands Queries

9. Adding things
• In the IoT, both things and users are
updating the command model
• How do we scale up and change
thing interaction from user
interaction independently?
• What is the appropriate model for
the write storage?
• How/where can we make use of
time series storage?
Domain
Logic
Write
Storage
Generate
Views
Read
Storage
Query
Generation
Commands
API Presentation
UsersThings
Time
Series
Store

10. Event Sourcing
• Instance of CQRS where commands
validated and processed to create events
• Every state change (event) is captured
and stored in sequence in an Event Store.
• Other processes can subscribe to events.
• Characteristics
• History – for free
• Replay events to regenerate the current state
• Or any point in time
• Can ‘fix’ state, e.g. By reversing past events.
User Presentation
Process
Commands
Event
Store
Process
Events
State
Store
Commands
Events
Query
Generation
Published
Events
Queries

11. Event Sourcing for IoT
• Two main sources of events –
users and things
• Separate thing ‘presentation’
• scaled out independently
• often # things > users
• Separate thing command
processing and generation
• Tap event stream for
• external systems
• Thing actuation
• time series storage
Actuation
User Presentation
Process
Commands
Event
Store
Process
Events
State
Store
Commands
Events
Query
Generation
Thing Presentation
Process
Commands
Generate
Commands
External
Systems
Sensing
Things
Events
ThingsThings
Gateways Things
Time
Series
Store
External IoT
Systems

12. Summary and Open Questions
• Event Sourcing architecture
• Can scale “thing presentation” and command processing independently
• However CQRS and event sourcing is more complex
• Methodologies, toolkits, frameworks available to help (ask Adaptech!)
• Remaining Open Questions:
• How do we handle the huge variety of protocols, services needed for a
complete IoT solution?
• How can we quickly get systems up and running? Moving from prototype to
production, support testing and simulation?
• How can we process data on the edge in gateways and devices for Fog
computing?

13. Node-RED and FRED for Event Sourcing
• Visual data flow programming –
rapid development
• Flows, nodes, wires and messages
(events)
• Huge variety of protocols and
services
• Can deploy NR flows on edge
devices and the cloud (FRED)

14. Why Node-RED?
• Almost every “new” thing in the IoT has a different protocol, platform
API today.
• Complete solutions often require pulling together device protocols,
online services
• Too much time spent on things like how to access GPIO pins on
devices, access serial port, set up HTTP or TCP/UDP endpoints,
connect to MQTT broker, OAuth dance on an API, connect to OPC UA
server…
• Node-RED wraps these issues in “nodes”, and lets developers focus on
solution at hand. In our case, getting commands and events in and
out of our system.

15. Where does Node-RED
fit?
• Simple Dashboards
• Thing Integration
• Device and gateway processing
• External System Integration
• Designing and Prototyping
systems
• Testing and simulation
User Presentation
Process
Commands
Event
Store
Process
Events
State
Store
Commands
Events
Query
Generation
Thing Presentation
Process
Commands
Generate
Commands
External
Systems
Things
Events
ThingsThings
Gateways Things
State
Store
External
IoT
Systems

16. FRED – Cloud-hosted Node-RED Demo

17. WIP: Event Sourcing with Node-RED
• Commands & Event processing
• Event Store integration
• REST API for User Commands & Queries
• Get latest state of things
• Send control commands
• MQTT interface for things
• Sensing
• Actuation
• Dashboard and Alerting
• Thing simulation

18. Summary
• IoT applications have unique characteristics
• Event Sourcing architectures are a natural fit for IoT applications
• Node-RED and FRED are useful tools
• Thing presentation & integration
• service integration
• rapid prototyping
• Dashboards
• Testing & simulation

19. Thank you! Questions?
Thanks to Adaptech Solutions for invitation and support!
FRED Platform: https://fred.sensetecnic.com (free to try):
• Cloud-hosted Node-RED
• MQTT service
• InfluxDb time series storage
Node-RED: https://nodered.org/
mike@sensetecnic.com
@mblackstock