2. 2
We build our computers
the way we build our cities -
over time, without a plan,
on top of ruins.
Ellen Ullman
3. About me
3
▸ Product Manager at Red Hat
▸ Former Principal Architect
▸ Committer at Apache Camel
▸ Author
・ Camel Design Patterns
・ Kubernetes Patterns
▸ @ bibryam
▸ https://www.ofbizian.com
4. About this talk
4
Application migration/modernization Rs
▸ Rehosting - lift-and-shift
▸ Replatorming - lift, tinker, and shift
▸ Refactor / Re-architect
・ Debezium
・ Apache Kafka
・ Kubernetes
5. Starting point
5
Challenges with monolithic applications
▸ Frequent deployment is difficult
▸ Obstacle to scaling development
▸ Scaling the application can be difficult
Expected modernization benefits
▸ Reduce time to market
▸ Greater team autonomy
▸ Improved automation
▸ Increased application performance
6. Target state
6
Cloud native microservices’ characteristics
▸ Independently deployable
▸ Modeled around a business domain
▸ Own their data
▸ Emit events (based on EDA)
▸ Built on open source technology such as
Kubernetes, Apache Kafka, Debezium
Measuring the results
▸ Lead time for change
▸ Deployment frequency
▸ Mean time to recovery
8. Strangler Pattern
8
High-level steps
▸ Identify functional boundary
▸ Migrate/reimplement functionality
▸ Synchronize data
▸ Reroute traffic to new service
▸ Decomposition old service
Main benefits
▸ No big bang migration
▸ Minimal changes to the monolith
▸ Low risk with the ability to fall back
9. Functional boundary
9
Functional boundary considerations
▸ Aggregates, bounded context based on DDD
▸ Event Storming technique by A. Brandolini
▸ Data model coupling
▸ Module dependencies
Where to start from?
▸ Easy win
▸ Noticeable improvement
▸ A representative effort
10. Interception options
10
Prerequisites
▸ Ability to identify inbound calls
▸ Ability to direct calls to new service (and back)
Routing options
▸ Client redirection
▸ Transparent HTTP proxy
▸ Shared protocol translation proxy
▸ Embedded/sidecar proxy
11. Interception options
11
Prerequisites
▸ Ability to identify inbound calls
▸ Ability to direct calls to new service (and back)
Routing options
▸ Client redirection
▸ Transparent HTTP proxy
▸ Shared protocol translation proxy
▸ Embedded/sidecar custom proxy
13. Data synchronization
13
Cons
▸ Polling delay, might miss updates
▸ Polling overhead
▸ Deletes not captured
▸ Not transparent to writing application
Pros
▸ Simple Installation and configuration
Cons
▸ Varying capabilities
▸ Limited integration capabilities
▸ Not portable
▸ Not easily testable
▸ Not transparent to source database
Pros
▸ No additional application required
Cons
▸ Requires specialized tools
▸ Requires database configuration
Pros
▸ All changes (deletes) are captured
▸ No polling delay or overhead
▸ Transparent to writing application
Triggers Log readers
Queries/Polling
14. Debezium
14
What is Debezium?
▸ Open source Change Data Capture platform
▸ Snapshotting, filtering, transformations
▸ MySQL, PostgreSQL, MongoDB, SQL Server,
Db2, Oracle, Vitess, Cassandra
▸ Use cases: data replication, cache updates, search index
updates, audit logs, sync data between services
15. Strangler Pattern with Debezium
15
Strangler Pattern with Debezium
▸ Transparent to the writing application
▸ Initial bulk import through snapshotting
▸ Schema, table, column filtering
▸ Single Message Transformations as anti-corruption layer
▸ Schema Registry for schema validation and
compatibility enforcement of data models
Debezium with Apache Kafka
▸ Guaranteed ordering, message compaction
▸ Pull based - re-read from start, or new changes
▸ Kafka Connect - offset tracking, fail over
16. Release steps
16
Steps so far
▸ Identified a functional boundary
▸ Migrated it as a new service
▸ Created a new data model in a new database
▸ Automated and deployed everything
▸ Bulk imported data and kept it in-sync
▸ No traffic routed to the new service
20. Release steps
20
Final steps
▸ STOP READ traffic to the legacy application
▸ Stop data synchronization to legacy system
▸ Decommission migrated module
22. Modernization challenges
22
Migration challenges
▸ Low-risk changes with demonstrable progress
▸ Deliver enhancements and new “business value”
▸ Simultaneously delivering new services
Distributed systems challenges
▸ Automation and operations at scale
▸ Dual-writes and long-running business transactions
▸ Analytical and reporting data needs
23. Operating event-driven services at scale
23
Microservices on Kubernetes
▸ Deployment/rollback
▸ Placement/scheduling
▸ Configuration management
▸ Resource/failure isolation
▸ Auto heal/upgrade (through operators)
Event-driven applications on Kubernetes
▸ Strimzi - operating Apache Kafka cluster
▸ KEDA - event-driven workload autoscaling
▸ Debezium - change data capture platform
▸ Knative Eventing - event-driven primitives
24. Outbox Pattern
24
Offers an approach for services to update their data store and
notify other services in a reliable and eventually consistent
manner.
▸ Addresses the dual-write problem
▸ Offers “read your own writes" semantics
▸ “at-least-once” semantics for consumers
25. Saga Pattern
25
Splits up long-running business transactions into a series of
multiple local transactions. When implemented using the Outbox
Pattern, it offers the benefits of orchestration and asynchronous
communication.
▸ Orchestration based coordination
▸ Asynchronous communication with Apache Kafka
26. Summary
26
Modern software systems are like cities - they evolve over time, on top of legacy systems. Using
proven patterns and standardized tools help create long-lasting systems that grow with your needs.
▸ Strangler Pattern - a low-risk application migration technique
▸ Outbox/Saga Patterns - reliable inter-service communication approach
▸ Debezium - a non-intrusive toolking for change data capture
▸ Kubernetes / Strimzi / Apache Kafka - de facto standards in their fields