Kafka Summit NYC 2019 Keynote by James Watters Senior Vice President, Strategy, Pivotal

1. James Watters
SVP Strategy, Pivotal

2. © Copyright 2019 Pivotal Software, Inc. All rights Reserved.
James Watters
April 2, 2019
Spring Boot+Kafka: the New
Enterprise Platform

3. Our Journey: Bring continuous delivery
with platform efficiency to enterprises.

4. New
Projects
Waterfall
ITIL
VMs
JavaEE
Batch
Products
Lean
CI/CD
Cloud Native Platforms
Spring Boot
Kafka Streaming Platform
Old

5. “For tech to be a real driver of innovation
and growth, IT needs to reorganize itself
around flexible platforms.” —McKinsey

6. Progress so far...

7. “We are happy to announce
that starting in 2018, Netflix is
also making the transition to
Spring Boot as our core Java
framework, leveraging the
community’s contributions via
Spring Cloud Netflix.”

8. Platform at
T-Mobile
SPEED
➔Time to Prod: 6 Months to weeks
➔# of deployments: 10x increase
SCALABILITY
➔Avg: 300M Transactions a Day
➔Peak: 1Billion Transactions a Day
STABILITY
➔43% reduction in app response time
➔83% fewer incidents, fixed 67% faster

9. IBM
IBM
Monolith
App
Cache
Win?
App

10. Full transformation required a
fundamental rethink….

11. “A business is a
series of events
and the reactions
to those events.”
—Jay Kreps, CEO, Confluent

12. “The only reason we don’t
think of events….is that so
far the technology has
trained us to think of data
as a static store.”
—Neha Narkhede

13. We restarted with
events first

14. What we’ve
learned
➔ Events are the language
bridge to the business
➔ This method of
identifying bounded
contexts is a secret to
decoupled architecture
➔ “Tell don’t ask”

15. Completing the Journey
IBM
App
Cache
App
Win? Microservices
Event-driven
Microservices
AppAppApp AppAppApp
AppAppApp

16. A platform built for a new way of thinking
➔Event + Microservice first
➔Team autonomy with
platform efficiency
➔Arbitrary scaling/scope
across enterprise
➔Turnkey multi-cloud

17. Streaming Data Platform

18. Immediate impacts

19. ➔ Global banking brand building greenfield
core banking and payments with Spring
Streams + Kafka
➔ Focus on microservices velocity required
platform first thinking
➔ Kleppman like view of Kafka: “We count
on Kafka for consistency, strict ordering,
replay, durability and auditability.”
➔ Cross cloud replication
A new continuously
delivered banking
platform

20. ➔ Turning moving packages into streaming
data with RFID, Kafka and Spring
Streams event based microservices
➔ Kafka, Kubernetes and Spring Boot in
every shipping center
➔ Multiple business microservices teams
can layer onto streaming platform to bin
pack last mile services.
➔ Prepared for unanticipated uses cases
Revolutionize our
shipping efficiency with
streaming microservices

21. PKS Managed Clusters
Messaging Middleware
Kafka
Binder Spring Data Repository
Event Driven Microservices
LTL Quote
Service
Scan RFIC
Services
RFID Triggered
Automation
Services

22. ➔ 100,00+ container build out of Spring
Streams, Kafka, key-value store
➔ Durability and consistency are critical
for potential legal actions
➔ Multi-phase stream processing with
Spring Streams leading to real-time
microservices alerting analysts
➔ Cross-cloud replication based on Kafka
➔ Continuously delivery required for real
time apps to improve accuracy and
functionality as project expands
Help secure a European
country?

23. Receiver
App
process queue
Fault tolerant
receiver pairs
staging
and
replication
Apps
Stream
Workers
Data
Enrichment
Stream
Workers
Data
Enrichment
Stream
Workers
Data
Enrichment
process queue
Stream
Workers
Data
Classification
Stream
Workers
Data
Classification
Stream
Workers
Data
Classification
buffer queue
S3 RAW Store
Receiver
App
X.000 Channel
Streams
RDBMS Store
3 DC
KAFKA
Replication
NoSQL
Store
Index
Store

24. ➔ Mainframe and monolithic RDBMS
data teams often the last to move to
continuous delivery
➔ CDC, Event Shunting, patterns
emerging allow streaming data
platform teams to offer mainframe and
legacy RDBMS events to microservices
teams
➔ “AirBnB Pattern” growing across
enterprises
➔ Each team can build appropriate
persistence and achieve multi-DC
replication with streaming platform
Let’s empower
pharmacy microservices
developers while
evolving our legacy?

25. PKS Foundation 1
PAS Foundation 1
Event Driven
Microservice
(Producers)
Event Driven
Microservice
(Consumers)
Domain
Driven
Microservice
Legacy
ODS
ODS
Kafka Cluster 1
Broker
MirrorMaker
cluster 1: topics cluster 1: topics
PKS Foundation 2
PAS Foundation 2
Event Driven
Microservice
(Producers)
Event Driven
Microservice
(Consumers)
Domain
Driven
Microservice
ODS
Kafka Cluster 2
Broker
MirrorMaker
cluster 1: topics cluster 1: topics

26. Spring Kafka ++
We are all in…
doubling down on a platform

27. Spring Cloud Stream
deals with the Kafka
scaffolding, so you
don’t have to
The power of Kafka streams
for developers
➔ Rapid on-ramp for Kafka Streams
consumption
➔ Simplifies construction of Event-
Driven Stateful Microservices
➔ Focus on your processing logic not
on configuration
➔ Full support of all Kafka streams
functionality

28. We are all in…
doubling down on a platform
Spring Kafka ++ PCC Kafka Integration Pivotal Function Service
(Knative ++ )
PAS Service Brokers
(Custom today)
PKS Confluent Operator
Testing and Integration
AppTX + Kafka CDC ++

29. Call to action
➔ Tim Berglund and Josh Long talk on
Spring and Kafka
➔ Contribute to GitHub features and issues
for Spring Stream
➔ Come visit us at the booth to talk
streaming microservices

30. #kafkasummit