O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Change Data Capture using Kafka

3.407 visualizações

Publicada em

This is the slide deck which was used for a talk 'Change Data Capture using Kafka' at Kafka Meetup at Linkedin (Bangalore) held on 11th June 2016.
The talk describes the need for CDC and why it's a good use case for Kafka.

Publicada em: Dados e análise
  • Entre para ver os comentários

Change Data Capture using Kafka

  1. 1. Change Data Capture Using Kafka Akash Vacher 2016/06/11
  2. 2. ▪ Akash Vacher Site Reliability Engineer Data Infrastructure Streaming (Bengaluru) Linkedin
  3. 3. Agenda ▪ Story of a webapp ▪ Problems ▪ Answer ▪ Proposed solution ▪ Why Kafka? ▪ Refined solution ▪ Q and A
  4. 4. Story of a Webapp
  5. 5. Dual writes
  6. 6. Problems
  7. 7. Inconsistent data stores
  8. 8. Atomic writes are a pain
  9. 9. The answer?
  10. 10. The humble log!
  11. 11. Database Replication
  12. 12. Proposed solution
  13. 13. The log should be… • Scalable • Durable • Low latency • High throughput • Allow bootstrapping – Have data persistence
  14. 14. Is Kafka a good fit?
  15. 15. Kafka has it all ▪ Horizontally Scalable ▪ Durable – Replication at Partition level ▪ Low latency ▪ High throughput ▪ Data is kept on disk ▪ Log compaction
  16. 16. Log Compaction
  17. 17. Revisiting proposed solution
  18. 18. Revised Solution
  19. 19. Change Data Capture
  20. 20. Questions?

×