Démystifier les architectures orientées événements avec Apache Kafka
•Transferir como PPTX, PDF•
0 gostou•178 visualizações
Les architectures orientées événements (ou EDA) sont perçues comme des entités magiques qui transforment instantanément vos systèmes en systèmes « temps réel » ! MAIS, en y réfléchissant, ne sont-ils pas déjà « temps réel » ? Je veux dire par là qu’ajouter un article dans un panier est pratiquement instantané dans la plupart des boutiques en ligne. En fait, une EDA résout un ensemble de problèmes totalement différents et, en faisant appel à Apache Kafka, nous allons suivre la voie de l’évolution (ou de la révolution). Les microservices sont faciles à prendre en main, mais une fois que c’est fait, on butte toujours sur les mêmes problèmes : l’accès aux données, la cohérence et les échecs (cela vous parle ? ). La solution ? Les modèles, les modèles, rien que les modèles… Vous avez déjà dû entendre parler des notions de « Event Notification », d’« Event-carried State Transfer » ou même d’« Event Sourcing », mais comment les utiliser pour résoudre vos problèmes ? Et, plus important, comment peut-on utiliser Apache Kafka pour tirer parti de ces modèles ? C’est ce que allons découvrir.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Démystifier les architectures orientées événements avec Apache Kafka
Semelhante a Démystifier les architectures orientées événements avec Apache Kafka (20)
Mais de Open Source Experience
Mais de Open Source Experience (20)
Démystifier les architectures orientées événements avec Apache Kafka
- 1. Bogdan Sucaciu Staff Engineer@ 8x8 |Pluralsight Author Demystifying Event-Driven Architectures with Apache Kafka bsucaciu.com @bsucaciu linkedin.com/bogdan-sucaciu
- 2. The Monolith A lot of code goes in here
- 4. The Monolith
- 5. SOA
- 9. Challenge -> Solution -> New Challenge GOOD BAD Data Access Scalability
- 11. Event-Driven Architecture Software architecture paradigm promoting production, consumption of, and reactionof events - Wikipedia
- 12. EDA
- 14. EDA
- 15. EDA Producer Consumer Event Stream Event Stream Event Stream Consumer Producer / Consumer Producer / Consumer
- 16. EDA Producer Consumer Event Stream Event Stream Event Stream Consumer Producer / Consumer Producer / Consumer
- 17. EDA Producer Consumer Event Stream Event Stream Event Stream Consumer Producer / Consumer Producer / Consumer
- 18. EDA Event Stream Logs Topics Queues WebSockets Reactive APIs Persisted Events Reactive Events
- 19. EDA with Apache Kafka KafkaBroker Inversion ofControl Fan-out 1 P-type/ EventStream Producersdon’tknow Consumers Topics
- 21. Challenge -> Solution GOOD BAD Data Access Scalability
- 22. Challenge -> Solution -> New Challenge GOOD BAD Data Access Consistency Scalability
- 23. The key to building successful EDA is understanding and accepting EventualConsistency
- 24. Past Present Future Query Command Event A B Message A B A ? A B A ?B A B
- 26. Event Producer Event Consumer privatestaticString TOPIC = "event-notification"; … Propertiesprops=new Properties(); props.put(BOOTSTRAP_SERVERS_CONFIG,"broker:9092"); props.put(KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); props.put(VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); KafkaProducer<String,String> producer= new KafkaProducer<>(props); ProducerRecord<String,String> producerRecord= new ProducerRecord<>(TOPIC,key, value); producer.send(producerRecord); privatestaticStringTOPIC= "event-notification"; … Propertiesprops=new Properties(); props.put(GROUP_ID_CONFIG,"event.consumer"); props.put(BOOTSTRAP_SERVERS_CONFIG,"broker:9092"); props.put(KEY_DESERIALIZER_CLASS_CONFIG,StringDeserializer.class); props.put(VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class); KafkaConsumer<String,String> consumer= new KafkaConsumer<>(props); consumer.subscribe(Collections.singletonList(TOPIC)); while(true){ ConsumerRecords<String,String> records= consumer.poll(ofMillis(100)); records.forEach(record-> processRecord(record)); }
- 27. Producer Consumer 3 Consumer 2 Consumer 1
- 28. Producer Consumer 3 Consumer 2 Consumer 1
- 30. Address Changed Event P Kafka Broker C1 C2 C3 Only newaddress? Old & new address? Userinfo? ?
- 31. Address Changed Event P Only newaddress C1 C2 C3 Kafka Broker
- 32. Address Changed Event P C1 C2 C3 Kafka Broker
- 33. Address Changed Event P C1 C2 C3 Kafka Broker
- 34. Address Changed Event P Tax C2 C3 Kafka Broker X
- 35. Avoid querying back the source system whenusing events!
- 36. Avoid querying back the source system whenusing events! Events stillmake sense after 10 years in a datalake!
- 37. Distributed Source of Truth
- 38. Event Sourcing
- 39. Event Sourcing Initial Amount 0 Add 500 Add 200 Subtract 300 0 Current Amount
- 40. Event Sourcing Initial Amount 0 Add 500 Add 200 Subtract 300 Current Amount 500
- 41. Event Sourcing Initial Amount 0 Add 500 Add 200 Subtract 300 Current Amount 700
- 42. Event Sourcing Initial Amount 0 Add 500 Add 200 Subtract 300 Current Amount 400
- 43. GIT
Notas do Editor
- Introduction: name, company Myth: Event-driven architectures are all about being real-time / ETL There are specific use cases for real-time data such as fraud detection ( atm example ), sensors reading ( smoke alarms ) EDA solves a different kind of problems, something that is much more powerfull Theory is good, practice is even better: Apache Kafka Start with the beginning
- I think that everyone knows about monoliths It’s a simple but powerfull pattern All code is encapsulated into a single running process Usually, a database is attached to it ( hundreds of tables ) Monoliths are great but, there is a but
- All problems condenses -> scalability Application scalability: the app must server more and more requests, increase instance Team scalability: 20-30 teams working on the same code repository; a lot time wasted due to communication
- - Some very smart persons come up with the idea of splitting up the monolith
- Service oriented architecture Code is split into multiple modules It was hard to properly define the borders of each service
- Domain-driven microservices Domain -> sub domains -> bounded contexts -> aggregates -> cool stuff Each microservice should have ownership over data Different databases: relational, nosql: document-based, key value stores
- Start monolith -> solves
- In software engineering we don’t have good or right, bad or wrong, we have tradeoffs. New challenge
- It is really hard to access the data you need
- Data is encapsulated by each microservice One customer makes a request -> cascade of other requests to aggregate data Challenges Coupling Event-driven architectures to the rescue Haven’t told what event-driven architectures are - change of paradigm from data to events Microservices are typically data driven: each microservice encapsulates data Events on the other hand promote a concept called data liberation
- events is the centerpiece definition
- - simplify
- -make it unidirectional
- -transpose it event-driven
- New concept: event stream Event stream -> middleware to facilitate transport of events 2 types of applications: producers and consumers
- Create and send events
- Retrieve and process events
- Protocols and tools Anything that you can asynchronously subcribe to Examples Reactive events: once consumed, they can never be brought back; Persistsed events: if something fails during consuming process, can replay
- Event streams as topics in Kafka 4 Key observations Inversion of control -> spring example; lis a good thing; REST-based producer pushes data, event-driven consumers has to retrieve it; producer only pushes it to the event stream Fan out -> multiple consumers can retrieve events from the same event stream; data is accesible; it is there to be taken 1 producer type per event stream -> difference between event stream and a topic; a topic can persist anything; data structure must be in place Producers dont know consumers: from application perspective; architecture perspective everything is well known
- Event-driven Architectures solves the data access problem
- Software engineering is about tradeoffs
- Syncronous APIs -> we got used to Strong Conistency
- Microservices have strong consistency Not all use cases can be made event-driven Typical system is a mix between event-driven and synchronous APIs Talk later about why eventual consistency matters What is an event?
- A message is just data exchanged between 2 services Data must have a structure A message may have differnet forms Patterns of EDA 2 types of events
- Action that has happened or has been triggered It must be a meaningfull action to the busines Cusomer Logins to our mobile application -> customer takes a photo Talk about e-commerce example; suggestions carousel Great example -> No response
- Very similar; Difference nuance compared to event notification
- It introduces coupling Let me say that again
- Data is scattered across multiple services Workaround: Copying parts of data from one database to another Challenges: -how to keep it in sync? How to make sure we have the latest versions of the data? Distributed systems needs another
- Most powerfull pattern, but also very demanding It all comes down to an event log
- Event log
- State is called materialized view Eventual consistency -> not all consumers retrieve the data at the same time Can always discard the entire database and rebuild it
- Barely scratched the surface: there are a lot of things to learn Change Data Capture, streaming pattern Thank you for joining this essions Feel free to get in touch with me