apidays LIVE Australia 2020 - Building Business Ecosystems Building an Enterprise Eventing Platform using Apache Kafka Gnanaguru Sattanathan, Solutions Engineer at Confluent, Kevin Barton Solution Designer at NAB & Mathew Chai, NAB

1. Building an Enterprise Eventing
Platform using Apache Kafka
From REST APIs and Message Queues, to Event-Driven Streaming
Gnanaguru (Guru) Sattanathan, Solutions Engineer - Confluent
Kevin Barton, Senior Consultant, Solution Design - National Australia Bank
Mathew Chai, Senior Consultant, Support - National Australia Bank

2. Enterprises (especially Banks) are moving from
software users, to becoming software
Faster
application
development &
time to market
Enable teams
to build
independently
Reduce
infrastructure
cost and
complexity
Eliminate
dependencies in
feature delivery
Scale services
with smaller,
agile teams
“In many ways, we see ourselves as a technology company with a banking license.” —
Michael Corbat, Citi CEO

3. Evolution of Monolith
APP APP APP APP
Monolith
Large self contained application
Complex app with highly interdependent parts
Poor developer experience and productivity
Slow feature delivery
Difficult to deploy, impacting other systems
Requires replication of entire application
Microservices
Multiple smaller, single function apps
Independently deployable and upgradable
Built around solving business capabilities
Can be built w/ different programming languages
Scalable and agile = faster feature delivery
Leverages newer development platforms
To Microservices

4. Existing approaches to developing microservices
Messaging Queues
(rabbitMQ, ActiveMQ)
● Message brokers act as a centralized
messaging service through which all
of the microservices communicate
● Brokers handle messaging queueing,
HA, reliable communication between
services
● Messages received in a queue rather
than dropped and processed later
Communication via
HTTP REST API
● Benefits:
○ Simple set up
○ Efficient message delivery
● Synchronous communication
○ Client sends request and waits
for response

5. Challenges with REST
based microservices
Fulfillme
nt service
Stock
service
Order
service
Return
service
Payment
service
UI service
GUI
● Difficult to enforce standards across
services
● Does not scale if servers are
synchronous
● Risky inter-service dependencies
● Services required to maintain state
● Complex management and version
compatibility—slows development
● Requires load balancing

6. Challenges with legacy
messaging queues
Microservice
Producer
Legacy
Messaging
Queue
Queue 1
Queue 2
Microservice
A
Microservice
B
● Extremely difficult to scale
● Lacks message persistence—if
message delivery fails, its
unavailable for replay
● Low throughput and high
latency
● Need prior knowledge of
consumers
● Expensive MQ and mainframe
technology costs
●

7. Fulfillment
service
Stock
service
Order
service
Return
service
Payment
service
UI service
GUI
Why build with
Confluent
● Completely decoupled microservices
● Single standard for
inter-communication
● Maintains version compatibility
● Asynchronous services
development
● State persistent on single platform
for replay
● Distributed and highly scalable
● Process data in flight and real-time
● Deployment flexibility -- on prem, in
cloud, or hybrid
Confluent enables a new class of event driven
microservices

8. Why does it make sense?
Microservices
Distributed
Semi-loose with API versioning
Requires a consistent distributed
system
Synchronous
Event Driven Microservices
Central, synchronized through events
Completely decoupled (Fire and forget)
The streaming platform reduces the
dependencies to external system
Reactive (asynchronous)
State
Coupling
E2E Testing
Execution Model

9. Fireside Chat
Kevin C Barton
National Australia Bank
Mathew Chai
National Australia Bank
Gnanaguru(Guru)
Sattanathan
Confluent

10. Workshop happening today !
Kafka streaming in minutes on Confluent Cloud: Create a
complete stack of fully managed services in Confluent Cloud
September 16, 1:50pm-2:40pm