The document discusses using Apache Kafka for security information and event management (SIEM). It describes some common problems with enterprise SIEM systems like handling large volumes of log data from various sources in different formats. Kafka is proposed as a solution because it offers high availability, scalability, and high throughput to ingest and process large amounts of log data. The document outlines an architecture where logs are ingested from various sources into Kafka topics, then filtered and routed using Kafka Connect and ksqlDB to transform and send the data to various destinations like Splunk. It also describes testing the solution's ability to process large data volumes and scale horizontally.
2. https://digitalis.io
2
Jason Bell
ABOUT
Working with Kafka since 2014, in
development, support and now DevOps.
Author of Machine Learning: Hands on
for Developers and Technical
Professionals, published by Wiley.
Kafka DevOps Engineer