[AerospikeRoadshow] Apache Pulsar Unifies Streaming and Messaging for Real-Time Data
https://www.eventbrite.com/e/real-time-data-evolution-forum-tickets-421382866497
Join us to discuss the latest innovations companies are adopting to meet the growing data demands for the just in time economy.
Timothy Spann, Developer Advocate, StreamNative
The Ludlow Hotel 180 Ludlow Street New York, NY 10002
hursday, November 3rd, 2022 - 12:00pm to 6:00pm (ET)
1:10pm - 1:30pm | Apache Pulsar Unifies Streaming and Messaging for Real-Time Data
Take a multi-cloud approach to your messaging and streaming
How to use Pulsar's dynamic scalability to right-size your cluster automatically
Time is Money: How Pulsar's single-digit latency will accelerate your business
Real-Time Fraud Detection at Scale with Apache Pulsar
Tim Spann, Developer Advocate, StreamNative
2. ● Apache Pulsar Committer | Author of Pulsar In Action
● Former Principal Software Engineer on Splunk’s messaging
team that is responsible for Splunk’s internal
Pulsar-as-a-Service platform.
● Former Director of Solution Architecture at Streamlio.
David
Kjerrumgaard
Developer Advocate
3. Tim Spann
Developer Advocate
Tim Spann, Developer Advocate at StreamNative
● FLiP(N) Stack = Flink, Pulsar and NiFI Stack
● Streaming Systems & Data Architecture Expert
● Experience:
○ 15+ years of experience with streaming technologies including Pulsar,
Flink, Spark, NiFi, Big Data, Cloud, MXNet, IoT, Python and more.
○ Today, he helps to grow the Pulsar community sharing rich technical
knowledge and experience at both global conferences and through
individual conversations.
4. Sijie Guo
ASF Member
Pulsar/BookKeeper PMC
Founder and CEO
Jia Zhai
Pulsar/BookKeeper PMC
Co-Founder
✓ Data veterans with
extensive industry
experience
✓ Original creators of Apache
Pulsar & BookKeeper
✓ Operated the largest
Pulsar/BookKeeper cluster
Matteo Merli
Pulsar PMC Chair,
BookKeeper PMC
CTO
StreamNative Executive Team
5. Apache Pulsar is a Cloud-Native
Messaging and Event-Streaming Platform.
6. CREATED
Originally
developed inside
Yahoo! as Cloud
Messaging
Service
GROWTH
10x Contributors
10MM+ Downloads
Ecosystem Expands
Kafka on Pulsar
AMQ on Pulsar
Functions
. . .
2012 2016 2018 TODAY
APACHE TLP
Pulsar
becomes
Apache top
level project.
OPEN SOURCE
Pulsar
committed
to open source.
Apache Pulsar Timeline
8. Pulsar Has a Built-in Super Set of OSS
Features
Durability
Scalability Geo-Replication
Multi-Tenancy
Unified Messaging
Model
Reduced Vendor Dependency
Functions
Open-Source Features
10. Fintech Powered by Pulsar
10
Low latency
Geo-replication
Data integrity
High availability
Durability
Multi-tenancy
Multiple data
consumers:
Transactions,
payment
processing, alerts,
analytics, fraud
detection with ML
Large data
volumes, high
scalability
Financial
event
messaging
Many topics,
producers,
consumers
11. 11
Designed for
teams, with
built in
multi-tenancy
Power and
flexibility,
w/ support for
simultaneous
streaming and
messaging use
cases
Ideal for
high-scale,
mission
critical
microservices
Easy to use,
with a simple
pub/sub API
Asynchronous APIs Empower All
12. Ideal for app and data tiers
Less sprawl and better
utilization
Cloud-native scalability
Build globally without
the complexity
Cost effective long-term
storage
Pulsar across the
organization
13. Joining Streams in SQL
Perform in Real-Time
Ordering and Arrival
Concurrent Consumers
Change Data Capture
Data Streaming
17. Background
● The third-largest payment
provider in China behind
Alipay and WeChat
Payment
● 500 million registered users
and 41.9 million active users
● Need to improve the
efficiency of fraud detection
for mobile payments
● Current lambda architecture
of Kafka + Hive is complex
and difficult to maintain
Benefits
● Reduce complexity by 33%
(clusters reduced from six to
four)
● Improve production
efficiency by 11 times
● Higher stability due to the
unified architecture
Why Pulsar
● Cloud-native architecture
and segment-centric
storage
● Pulsar is able to do both
streaming and batch
processing
● Able to build a unified
data processing stack
with Pulsar and Spark,
streamlining messy
operations problems