Whether you are a die-hard DC comic enthusiast, mad for Marvel, or completely clueless when it comes to comic books, at the end of the day each of us would love to possess the superpower to transform data in seconds versus minutes or days. But architects and developers are challenged with designing and managing platforms that scale elastically and combine event streams with stored data, to enable more contextually rich data analytics. This made even more complex with data coming from hundreds of sources, and in hundreds of terabytes, or even petabytes, per day. Now, with Apache Kafka and Intel hardware technology advances, organizations can turn massive volumes of disparate data into actionable insights with the ability to filter, enrich, join and process data instream. Let's consider Information Security. IT leaders need to ensure all company data and IP is secured against threats and vulnerabilities. A combination of real-time event streaming with Confluent Platform and Intel Architecture has enabled threat detection efforts that once took hours to be completed in seconds, while simultaneously reducing technical debt and data processing and storage costs. In this session, Confluent and Intel architects will share detailed performance benchmarking results and new joint reference architecture. We’ll detail ways to remove Kafka performance bottlenecks, and improve platform resiliency and ensure high availability using Confluent Control Center and Multi-Region Clusters. And we’ll offer up tips for addressing challenges that you may be facing in your own super heroic efforts to design, deploy, and manage your organization’s data platforms.

1. Intel and Confluent join forces to optimize the performance of enterprise data centers
Sandeep Togrikar
(Sr. Solutions Architect - Intel)
Bert Hayes
(Solutions Engineer – Confluent)

2. 2
Agenda
Introductions
Why Apache Kafka?
Confluent Key Features
Confluent & Intel Better Together
How to Get the Most out of Your Deployment
Recommended Reference Architecture
Call to Action

3. 3
Apache Kafka
60%
The Rise of
Event Streaming
Fortune 100 Companies
Using Apache Kafka
Apache Kafka is a distributed event streaming platform developed at LinkedIn
Please see Confluent press release for more information

4. 4
Kafka Empowered Data Centers
Data center with the power of Apache KafkaData center without the power of Apache Kafka
Brittle, fragile, difficult to maintain, difficult to
troubleshoot, difficult to add things cleanly
Data producers decoupled from consumers to simplify
✶ Intel Lightning Talk ✶
Building a Modern, Scalable Cyber Intelligence Platform with
Confluent
SIEM
Example Business Cases
Mainframe offload

5. Confluentfeatures
ksqlDB
Confluent
Replicator
Multi-Region
Clusters

6. 6
Multi-Region Clusters
Data Loss:
Prioritize durability
or throughput at the topic level
Recovery Time:
Measured in seconds
(dependent on # partitions)Disaster Event
Time
Why
• Limit rebalances to topics or replica-
placement only
• Ability to batch or cancel
Key features
• Reduces WAN tax
• Simplifies high availability for consumers
Improved tooling for faster failover/failback

7. ksqlDB
Why
• Simplify the developer’s mental model
for ksqlDB
• Create new topics without having to
work in the CLI.
Key features
• View a summary of all clusters
• Develop and run queries
• Support multiple ksqlDB clusters at a
time
Control Center ksqlDB editor
7
Easily build event streaming applications

8. Confluent Replicator
Why
• Selectively copy content to the cloud
• Streamlines bridge-to-cloud deployment
Key features
• Facilitates multi-cloud deployments
• Supports global federated data sharing
Source Kafka
DC1
Source Kafka
DC2
Kafka Connect
DC 1 Replicator DC 2 Replicator
Destination Kafka
Aggregate DC
Replication to an Aggregate Cluster
8
Replicate configured topics across sites

9. Confluent+INTEL

10. 10
Intel & Confluent Better Together
Build new solutions for
agile, data driven
intelligent enterprises
Contribute to innovative
open source technology
Collaborate with partners
across the ecosystem to service
various industry segments
CSPs
Working together to deliver a performance optimized event streaming platform on Intel® architecture—
on prem or in the cloud.

11. MOVE STORE
SOFTWARE & SYSTEM LEVEL
PROCESS
ETHERNET
Unleashing the Potential of Data
11
3D NAND SSD
REPLICATOR BROKER ksqlDB

12. Kafka Solution Architecture
12
Management & Monitoring
Control Center | Security
Enterprise Operations
Replicator | Auto Data Balancer | Connectors | MQTT Proxy | k8s Operator
Data Compatibility
Schema Registry
Development & Connectivity
Clients | Connectors | REST Proxy | kSQL
Apache Kafka
Core | Connect API | Streams API
Commercial Software
Community Software
Confluent Platform
Datacenter Public Cloud Confluent Cloud
Customer self-managed Confluent fully-managed
Control Center (C3)
2S Intel® Xeon® Gold 6238 28C
Intel® Optane™ SSD
Memory 384GB (12 x 32GB)
kSQL
2S Intel® Xeon® Gold 6238 28C
Intel® Optane™ SSD
Memory 394GB (12 x 32GB)
Brokers
2S Intel® Xeon® Gold 6238 28C
Intel® Optane™ SSD
Memory 192GB (6 x 32GB)
Please see Disclaimers for full details and configs.

13. Throughput
Rate of data
movement
Latency
E2E elapsed /
response time
Optimize
Your
Deployment
13
Superpower Performance
Durability
Minimize lost
messages:
Availability
Minimize downtime /
Recover ASAP
Optimal performance involves trade-offs between throughput, latency, durability, and availability.
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks.​

14. Results when Tuned for Sub 3ms Latency on 10Gbps Network
With15:8 io:network thread
config setting
With 7:4 io:network thread
config setting
When using Intel® Optane™ SSD
over standard NVME
500MBPs 1GBPs
14
6.5x
Throughput
Please see Disclaimers for full details and configs.
more durable
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks.​

15. Recommended Production Deployment
DC-1
Offline Replicas
DC-2
Leader ISR
15
Brokers:
Compute: 2S Gold 28C
Storage: 4x Optane 750GB
Memory: 192GB
Networking: 10G
Supporting Nodes:
Compute: 2S CLX Silver 10C
Storage: 2x1.9TB(OS)
Memory: 32GB
DC-3
Broker 1
Broker 2
Broker 3
Zookeeper 1
Zookeeper 2
Broker 1
Zookeeper 3
Zookeeper 4Zookeeper 5
Single Kafka Cluster
Producers Consumers
Streams Apps
ProducersConsumers
Brokers:
Compute: 2S Gold 28C
Storage: 4x Optane 750GB
Memory: 192GB
Networking: 10G
Supporting Nodes:
Compute: 2S CLX Silver 10C
Storage: 2x1.9TB(OS)
Memory: 32GB
Streams Processor:
Compute: 2S Gold 28C
Storage: 2x P4510 4TB
Memory: 384GB
Control Center
Compute: 2S Gold 28C
Storage: 2x P4510 4TB
Memory: 384GB
Networking: 10G
Multi-region w/site failover
Broker 4
Broker 5
Broker 2
Broker 3
Broker 4
Broker 5

16. 16
Optimize Enterprise Data Center Performance
Questions? Reach out to us.
Sandeep Togrikar , Sr. Solutions Architect - Intel
sandeep.togrikar@intel.com
Bert Hayes, Solutions Engineer – Confluent
bhayes@confluent.io
1 For streams processing, choose Apache Kafka.
2 For Apache Kafka for the enterprise, choose Confluent.
3 For Confluent on prem or in cloud, choose Intel®.
4
For more details see Joint Reference Architecture & IT@Intel InfoSec white paper
(coming soon)