Discover concrete architectural patterns to use in-memory systems (like Infinispan, Redis etc) where it makes a difference. They are versatile tools that can be used in a myriad of ways but unlike duct tape and WD-40, it’s not intuitive when and how to use them.

1. Practical architecture
In-memory data systems patterns
WD-40 between apps and data
Emmanuel Bernard
Red Hat

2. Emmanuel Bernard
@emmanuelbernard

3. Data is coming
Data is flowing

4. Insightful application
Observe - collect
data
Orient - Put in
context
Decide - select
course of action
Act - Do it
Continuous improvement loop
Data services
Event stream
Data analysis
Iterative storage
Reasoning, decision
models and
optimization
Execution engine

5. In-memory systems
an case

6. Key foundation
key/value store
Distributed
Elastic
Optional schema
ASL 2.0

8. in-memory systems
in an architecture

9. Caching
Caching
service
data
Get faster
Hot cache
App 1
App 2
Infinispan
Datastore

10. Shared memory
Stateless apps in a
stateful universe
Microservice
“Stateless" “cluster”
Infinispan
SharedMapSharedMapSharedMapSharedMap
Load
balancer

11. Primary store
Microservice
“local” storage
Derived store
(Fast) Search capabilities
Analytics
Distributed tasks
App 1
App 2
Infinispan

12. Distributed stream
Move computation to data
map(λ)
λ
λ

13. In action!

14. Station board
stop
station
train

15. What is the time of the day when
there is the biggest ratio of
delayed trains?

16. Mixing stores
Mix traditional datastores
and in-memory
Temporary data
Heavy computation
App JPA
OGM
ORM
App JPA
OGM
ORM
App JPA
OGM
ORM
Infinispan
RDBMS

17. Data virtualisation duct taping:
teiid.org
Microservice 1
Big old
Datasource
Dedicated
Schema 1
Dedicated
Schema 2
Dedicated
Schema 3
Data Virtualisation
Microservice 2 Microservice 3 Old app

18. Data layering

19. Analytics and Spark
Computation out of Spark
Source of data
Source of events
Immutable RDD
Fast RDD caching
Spark cluster
Infinispan
Cassandra
Hive
RDBMS
App 1

20. Syncing grids
Active / Active
Data synced between sites
One grid per usage
OLTP
Analytics
App 1
App 2
Infinispan
Infinispan
Active / active
cross site replication
Datacenter 1
Datacenter 2
Load
balancer

21. Pay attention, be reactive
Reactive application
Reactive microservice
Also look at Debezium
App 1
“Reactive app” 2
Infinispan
Change event
“Reactive app” 1
Change event

22. Continuous query
Reactive to data changes
Index the query, not the data
from User u where u.age >18 and u.country = ‘France’
from User u where u.age >18 and u.hasSnapshat = true
from User u where u.email contains ‘yahoo.com’
{ “country”: “France”,
“age”: 17,
“email”: “e@gmail.com”,
“hasSnapshat”: true }
{ “country”: “France”,
“age”: 19,
“email”: “e@gmail.com”,
“hasSnapshat”: true }
=>

23. In action!

24. CentralTrain Delays Dashboard

25. Debezium and CDC
Change Data Capture
Transforms datastores into data change events
No change to existing apps and datastores
Add features as separate apps
Decorrelate event consumers (each at its own pace)

26. Data are like Gremlins but good
Observe - collect
data
Orient - Put in
context
Decide - select
course of action
Act - Do it
Continuous improvement loop

27. Thanks
http://infinispan.org
http://blog.infinispan.org
@infinispan
https://github.com/infinispan-demos/
swiss-transport-datagrid
http://debezium.io
http://teiid.org
Explore your data
Take calculated risks
Any use case?

28. Credits
engineer by Wilson Joseph from the Noun
Project
panel by gira Park from the Noun Project
Approve by Aha-Soft from the Noun Project
Database sharing byYuguDesign from the Noun
Project
ram by Andrea Rizzato from the Noun Project
Database Search by Nimal Raj from the Noun
Project
Cloud Analytics by Kevin Augustine LO from the
Noun Project
Broken Computer by Dan Hetteix from the
Noun Project
data search by Gregor Črešnar from the Noun
Project
Server by Creative Stall from the Noun Project
Network by Creative Stall from the Noun
Project
transformation by Felipe Perucho from the
Noun Project
analytics by Roman Kovbasyuk from the Noun
Project
Server by Designify.me from the Noun Project