Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Enabling a Real-Time, Agile, Event-Driven Enterprise
1. Enabling a Real Time,
Agile, Event-Driven
Enterprise
Sumeet Puri
SVP, Field Tech, Solace Corp
Gartner AADI - 2nd March 2020
2. Payments Evolution
HI GH VA LUE
LOW VOLUME
T+3 SE TTLE ME N T
SOA , A T BE ST
SI LOE D A MON GST
CON SUME R S
MULTI Y E A R P R OJ E CTS
HI GH VOLUME
LO W VA LUE , MO BI LE
R E A L TI ME
E VE N T D R I VE N
FOD D E R FOR A I /ML
A P I OR I E N TE D
A GI LI TY N E E D E D –
I MP LE ME N T QR COD E
I N W E E KS
2015 2020
4. But, most
companies’ IT
systems don’t
meet the needs of
modern digital
business.
Sources from : Gartner Application Architecture, Development & Integration Summit 2018
7. The Universe is event driven... We hardly
“poll” for things but almost always react to
events
• An occurrence
• An API Call
• A sensor trigger
• A phone app click
• A browser click
• A rule generated alert
• Each step of provisioning
• A notification
• Anything other than polling to check for
updates
What is an
Event?
10. • Cloud Native
• API First
• AI/ML Enablert
• IOT Backbone
• Cloud native
• Robust
• Burst Proof
• Faster time to
market
• Microservices,
Choreography
• Better response
times
• Real time
• Push vs pull
Better CX
Better
Agility
Innovation
Backbone
Future
Proof
Why
Event Driven?
Agility
Real Time
Responsiveness
11. Source: Gartner "Event-Driven Computing for Digital Business Technology Leaders“
AADI Summit London, 20 May 2019
Gartner Says Events Turn IT into an Engine
11
13. BatchPolling Event Streaming
Long Time Real Time
Event Driven & APIsETL → SOA &
APIs
Event-Driven is the new imperative…
Liberate Data at Rest - Identify Real Time Candidates
Ask the question – why not real time? Business, or IT problem?
Select high visibility, quick win candidates to start the event driven journey
14. Ok, has the Event Streaming
paradigm never been used before?
Can we be “inspired”?
15. The Un-named Event Mesh – Trading platforms
THE S OL A CE HER I TA GE
18μs latency 99.999% uptime 28M msg/sec
Pub/Sub & Queuing Zero Data Loss Multi Protocol
WAN Optimization Burst Handling Hybrid Cloud
Often replaced TIBCO
EMS, RV, Kafka, IBM
MQ, RabbitMQ,
ActiveMQ, ZeroMQ,
Kaazing, WebMethods,
SoftwareAG UMS,
29W…
And
many
more!
Event Driven for 10+ years!
16. Who is Solace?
World’s foremost experts on event
distribution and messaging middleware
‐ 21 patents
‐ 9 of top 12 Investment banks
‐ Top American & Indian telcos
‐ Air traffic management around the world
HQ and Engineering
in Ottawa, Canada
15 Global Offices
Presence in Americas, AsiaPac, Europe
Award-
winning
business
28. The SOA Way
B2B
Heavy ESB at the heart of all services Orchestration |
Transformation | Adaptation
Corebusiness
Logic
Legacy
Proprietary
SOAP/HTTP
Service
Consumers
What worked:
• Better than Client Server
• Some Reusable Services
• Some Standards with
SOAP/HTT or JMS
• ESBs as fast dev tools
• Some Governance
Challenges:
• Runtime Bottleneck – ESB in
the middle of everything
needs to be scaled all the
time!
• Serial Communication,
everything inline, often not
parallel
• Lack of Agility – slot to
market
• ESB team is often the
bottleneck
• ESB team far from SME
• Complex Orchestration
hard to maintain
• Minimal Reuse – services
often too coarse grained
• Forced changes when new
components added, such as
Big Data ingest
SOAP/JMS
Messaging (JMS)
ServiceEnablementvia
ESB
29. Digital Transformation –
Evolving SOA to Event Driven Architecture
Invert the ESB. Messaging evolves to an Event Mesh
Choreography via Topic Routing over the Event Mesh
30. Evolution of Messaging and Integration
Producer
A
Consumer
A
MQ
Producer
B
MQ
Producer
C
MQ
Producer
A
Consumer
C
Producer
B
Producer
C
IBM MQ Point to Point
• Technical Interfaces – Producer
needs to know the exact end
point of the Consumer
• No grain for functional
capability
• Point-to-point model
Enterprise Service Bus
• Functional capability – Producer
needs to know the exact end point
for a feature and not Consumer
• Coarse grain functional capability
• Pub-Sub Model
Event Driven Architecture
• Dynamic Routing on Taxonomy –
Producer need not know end point of
system or feature; he needs to go to
Event Mesh (fabric that connects all
end points)
• Fine grain functional capability
• Event Model (pub-sub+)
Producer
A
Consumer
B
MQ
Producer
B
MQ
Producer
C
MQ
Producer
A
Consumer
C
MQ
Producer
B
MQ
Producer
C
MQ
Producer
A Consumer
A
Producer
B
Producer
C
Consumer
B Producer
A
Producer
B
Producer
C
Consumer
A
Consumer
B
Consumer
C
32. • Order to Fulfillment
• Manufacture to Supply
• Payment Processing
• Resource Management
• Data Lake Ingress and Egress – Data
lake or event stream?
• New Cloud Native or Hybrid Cloud
Projects, such as Employee
Onboarding
• IOT Platform as a Service
Examples as
thought process
catalysts
All the above, and many more candidates are a
great way to start your event driven and API
(streaming API) journey
36. Event Driven
Foundation:
Tools and Techniques
Event Mesh
•Event Brokers for publish subscribe routing of events with all
qualities of service
•Open standards
•Event Sourcing and Replay
Event Portal
•Design and Runtime tool for event cataloging, choreography,
code gen, event discovery and metrics
Event Taxonomy
•Routing rules in the form of topics which form the meta data for
events, used by the event mesh implicitly to route events
OrganizationalAlignment
•Align the middleware team and LOB teams and providers of
infrastructure, CoE, and custodians of the event catalogue
•Align partners and developers for delivery
37. Event Taxonomy – Wildcards are Critical
• Event driven APIs are described by “Topics”
– Topic routing has been aroundfor 20 years and scales better than SOA
• Capital markets to IoT runs on this
– Topic routing is standardized
– Topics are routable via an Event Meshh
Message
Payload
Binary or json or xml, or anything!
REST URL : http://host:port/order/v1/postpaid/plan123/iphoneX
Event over REST
Message
Payload
Binary or json or xml, or anything!
Topic: order/v1/postpaid/plan123/iphoneX
Event over MQTT/JMS
Event Routing Enabled Taxonomy
REST JMS
Publish:order/v1/postpaid/plan123/iphoneX
Subscribe –all these will match:
order/v1/postpaid/>
or
order/*/*/*/iphoneX
*
41. • Microservices = Fine Grained
Functionality
• Event Driven Microservices
respond to events, produce
events
• Daisy Chain or Parallel
Integration via Event Mesh
• Open Standards – not just hard
wired REST
• Hard Wired REST gives you a
Distributed Monolith
What are
vent-Driven
Microservices?
Microservice
BusinessLogic
Listens to and
“responds” to
events
“responds” as new
events. In other
words, produces
events as a result
42. Event Driven Microservices
B2B
EventDriven
Microservices
Postpaid Order Management
JMS Topic:
order/v1/postpaid/>
AMQP Topic : order/v1/postpaid/plan123/iphoneXEvent
Producers
Publish Events:
• Topics/URL describe the event and
have other metadata to route
• Events are published to context
based, “/” separated URLs
• “/” tokenized string is used for
event routing
• ACLs apply for publishing as well as
subscribing, as to which topic/URL
a client can publish or subscribe to
for governance
Subscribe to Events:
• Event URLs or Topics taxanomy is
governed, catalogued
• Topics are creating only on runtime
as subscriptions – no design time
creation needed
• Microservices subscribe to events
of interest using topic subscriptions
• Persistent, Request Reply are also
supported.
Benefits:
• Latency of Event routing is in micro
seconds
• Throughput is in Millions/second
• Multiple protocols and translation
between them
Messaging–
EventMesh
REST URL: http://host:port/order/v1/postpaid/plan123/iphoneX
• Events are published to context based, “/”
separated URLs
• “/” tokenized string is used for event routing
• ACLs apply
• Microservices subscribe to events of
interest using topic subscriptions
• Persistent, Request Reply are also
supported.
• Wildcard subscriptions supported
New channels added
easily, just publish on
the same event
taxonomy
Event Mesh
Micro
Service
47. Move ESB use cases to
the edge – closer to
source and destinations
48. Event Mesh
ESB/Adapters
Event Driven Microservices – ESB at the Edge
API Platform
ESB/Adapters
B2B
Lightweight ESB at the edge (only if needed) - Orchestration |
Transformation | Adaptation
EventDriven
Microservices
JMS Topic:
order/*/*/iphoneX
iPhoneX Campaign AnalyticsPostpaid Order Management
Legacy
Proprietary
Standards
JMS Topic:
order/v1/postpaid/>
AMQP Topic : order/v1/postpaid/plan123/iphoneXEvent
Producers
Characteristics:
• As Event Driven Microservices are
Self Orchestrating, and ESB is not
required in most flows
ESB Challenges:
• ESBs slow things down, extra
layer
• ESBs don’t always add value, in
an ideal world there should be a
standardised canonical data
model, so makes sense to aspire
for it
• ESBs however are needed for
legacy integration and
orchestration
ESB at the Edge:
• So the best place for ESBs is
close to the source or destination
of data
• This way, interfaces are
quickly converted from legacy
to standards based
• ESB code is implemented by
folks who know business logic
– agile and quick
• ESB is basically used as a fast
dev tool
• Message Bus serves the Bus
based integration instead
Messaging–
EventingBackbone
REST URL : http://host:port/order/v1/postpaid/plan123/iphoneX
Micro
Service
Micro
Service
49. Event Mesh
Event Driven Microservices – ESB opportunistically changed to
microservices too!
API Platform
B2B
Lightweight Microservices at the edge -
Choreography | Transformation | Adaptation
EventDriven
Microservices
JMS Topic:
order/*/*/iphoneX
iPhoneX Campaign AnalyticsPostpaid Order Management
Legacy
Proprietary
StandardsJMS Topic:
order/v1/postpaid/>
AMQP Topic : order/v1/postpaid/plan123/iphoneXEvent
Producers
Characteristics:
• Powered by Solace
• Event Driven Microservices
• Topic Routed
• Millions of events per second
• Secure
• Multi, Hybrid Cloud enabled
• Elastically Scalable
• Pluggable caching
Benefits:
• Very High Performance
• Burst Handling – never miss an
order!
• Parallel event delivery– faster
response time
• Much improved Agility – faster
time to market
• New microservices just start
subscribing to events!
• ESB only by exception, not
norm - ESB changes minimized
• ESB moves closer to
source/destination logic
• Complex Orchestration to
Subscription based Self
orchestration
• Better Governance – Event
Based (topic based)
Messaging–
EventingBackbone
REST URL : http://host:port/order/v1/postpaid/plan123/iphoneX
Micro
Service
Micro
Service
Micro
Service
Micro
Service
Micro
Service
Micro
Service
51. Multi Cloud Event Mesh
API Platform
EventDriven
Microservices
Microservice
Event
Producers
Messaging–
EventingBackbone
Microservice
Micro
Service
Micro
Service
52. • Solace Event
Streaming Platform
is an end to end
foundational
architectural layer
for SAP
Modernization, IOT,
Event Driven APIs
and Event Driven
iPaaS
Hybrid Multi Cloud Event Mesh
Data Lake, AI, ML
Retail Factories
Tracking
IOT API
iPaaS
ESB