Kafka on Kubernetes
•
0 gostou•165 visualizações
Victor Gamov from Confluent presented 'Streams must fFlow: Developing fault tolerant stream processing application with Kafka Streams and Kubernetes’ at Montreal's very first Cloud Native Day, which took place on June 11, 2019.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Kafka on Kubernetes
Semelhante a Kafka on Kubernetes (20)
Mais de CloudOps2005
Mais de CloudOps2005 (20)
Último
Último (20)
Kafka on Kubernetes
- 1. @gamussa | #ImCloudNative | @ConfluentINc Kafka on Kubernetes: Does it really have to be «The Hard Way»? June, 2019 / Montreal, CA, 2019 @gamussa | #ImCloudNative | @ConfluentINc
- 2. @gamussa | #ImCloudNative | @ConfluentINc 2
- 3. @gamussa | #ImCloudNative | @ConfluentINc 3 I build highly scalable Hello World apps
- 4. Raffle, yeah 🚀 Follow @gamussa @confluentinc 📸 🖼 👬 Tag @gamussa With #ImCloudNative
- 5. @gamussa | #ImCloudNative | @ConfluentINc 5 A company is build on DATA FLOWS but All we have is DATA STORES
- 6. @gamussa | #ImCloudNative | @ConfluentINc 6 Pre-Streaming
- 7. @gamussa | #ImCloudNative | @ConfluentINc 7
- 8. @gamussa | #ImCloudNative | @ConfluentINc 8 New World Streaming first • DB/DWH + Many more distributed data systems • Monolith -> Microservices • Batch -> Real-time
- 9. @gamussa | #ImCloudNative | @ConfluentINc 9 Evolution of #devkafkaops Shell scripts ansible/chef Docker Kubernetes
- 10. @gamussa | #ImCloudNative | @ConfluentINc 10 https://twitter.com/claesmogren/status/1108456688175079424
- 11. @gamussa | #ImCloudNative | @ConfluentINc 11 🙋
- 12. @gamussa | #ImCloudNative | @ConfluentINc 12 Who run stateless workloads in Kubernetes? Who thinks it’s a good idea? Who run stateful workloads in Kubernetes? Who thinks it’s a good idea? 🙋
- 13. @gamussa | #ImCloudNative | @ConfluentINc 13 kafkaesque world of Kafka on Kubernetes
- 14. @gamussa | #ImCloudNative | @ConfluentINc 14 Well, it’s tricky © Translating an existing architecture to Kubernetes External access to brokers and other components Persistent Storage options on prem and clouds Security Configuration and Upgrades #devkafkaops
- 15. @gamussa | #ImCloudNative | @ConfluentINc 15 But I just want to deploy kafka
- 16. @gamussa | #ImCloudNative | @ConfluentINc 16 Workloads Deployment
- 17. @gamussa | #ImCloudNative | @ConfluentINc 17 Helm Charts
- 18. @gamussa | #ImCloudNative | @ConfluentINc 18 Helm Charts
- 19. @gamussa | #ImCloudNative | @ConfluentINc 19 Helm Charts
- 20. @gamussa | #ImCloudNative | @ConfluentINc 20 https://cnfl.io/helm_video
- 21. @gamussa | #ImCloudNative | @ConfluentINc 21 Helm Charts is just a GO Templates. How Charts help with rolling update?
- 22. @gamussa | #ImCloudNative | @ConfluentINc 22 Rolling Upgrade Kafka Broker Upgrades: 1. Stop the broker, upgrade Kafka 2. Wait for Partition Leader reassignment 3. Start the upgraded broker 4. Wait for zero under- replicated partitions 5. Upgrade the next broker
- 23. @gamussa | #ImCloudNative | @ConfluentINc 23 We will use StatefulSets with OrderedReady
- 24. @gamussa | #ImCloudNative | @ConfluentINc 24 We need SRE / Operator knowledge to manage the platform. You need Operator!
- 25. @gamussa | #ImCloudNative | @ConfluentINc 25
- 26. @gamussa | #ImCloudNative | @ConfluentINc 26 Show me your Operator
- 27. @gamussa | #ImCloudNative | @ConfluentINc 27 Rider Driver 2 Web Apps powered by a microservice each Apps are a ride sharing driver and rider location tracking Rider hails a ride and the driver drives towards the rider Real time location events powered by Kafka on Kubernetes KLyfft - Kafka app on Kubernetes
- 28. @gamussa | #ImCloudNative | @ConfluentINc 28 Demo
- 29. @gamussa | #ImCloudNative | @ConfluentINc 29 DO KAFKA ON KUBERNETES DEMO AND EVERYONE LOOSES THEIR MIND
- 30. @gamussa | #ImCloudNative | @ConfluentINc 30 Confluent Operator - Automated Security Configuration SASL PLAIN and Mutual TLS Authentication Automate configuration of truststores and keystores with secret objects Automate configuration of Kafka and all Confluent Platform Components
- 31. @gamussa | #ImCloudNative | @ConfluentINc 31 Be like Justin!
- 32. @gamussa | #ImCloudNative | @ConfluentINc 32 Rolling Upgrade Kafka Broker Upgrades: 1. Stop the broker, upgrade Kafka 2. Wait for Partition Leader reassignment 3. Start the upgraded broker 4. Wait for zero under- replicated partitions 5. Upgrade the next broker
- 33. @gamussa | #ImCloudNative | @ConfluentINc 33 Want to see advanced deployment? vs.
- 34. @gamussa | #ImCloudNative | @ConfluentINc 34 http://gamov.dev/ksummit_nyc19_k8s
- 35. @gamussa | #ImCloudNative | @ConfluentINc 35 GA Plans● We are in private Preview Release is over! ● 24 customers testing the Operator in Preview: ● Global customers ● Banks, Fin Tech, Retailers, Consumer Tech ● We are in the final stages of Preview and about to launch soon
- 36. @@gamussa | #ImCloudNative | @ConfluentINc Thanks! @gamussa viktor@confluent.io https://slackpass.io/confluentcommunity #kubernetes
- 37. 37