Testing distributed event-driven systems is hard, but you can only ship with confidence when you have robust integration tests. In this session, we explore how Testcontainers libraries allow you programmatically create, manage the lifecycle, and configure ephemeral instances of Kafka. From spinning up individual Kafka services to creating complex cluster topologies, your tests control the environment they require and run the same in any development environment or in CI. Whether you're working on the JVM, writing Go, .NET, or node.js apps, the Testcontainers approach gives your flexibility and confidence to reliably test your code. After this session, you'll learn enough to start testing your Kafka, Kafka Streams or Connect applications without breaking a sweat.

1. Better integration tests for
Kafka applications with
Testcontainers

2. Oleg Šelajev
Developer relations

5. Tests are essential

6. Unit tests
Integration
tests
e2e

8. Unit tests
Integration
tests
e2e

9. Timeline Local DBs
Integration Testing transformation
over the years (i)
Mocking VMs (Vagrant)

10. Timeline
Integration Testing transformation
over the years (ii)
Docker Compose
Docker Docker API

11. Common approaches to integration testing
In Integration tests we are interested in verifying the behavior and interactions of
multiple components.
For this purpose we can use:
● Shared instances
● Local installations
● In memory solutions
○ Mock server (Wiremock, Loki…)
○ In-memory service (h2, hsql...)
● Docker Compose

12. Testcontainers-java
• Created 7 years ago (Docker is 8 years old)
• github.com/testcontainers/testcontainers-java
• Uses docker-java API
• Integrates with frameworks, like Spring, JUnit
• Works with anything that runs in a Docker container

13. Growing ecosystem of modules

14. testcontainers.com/modules

15. testcontainers.com/modules

16. Why are Testcontainers - 🏆🏆🏆
●Flexible lifecycle API: isolation vs convenience
●Capable of abstractions
○ modules
●Readiness control
●Complex network topologies
●Randomized ports mapping
●Everything is dynamic allowing parallelisation
●Self-contained tests: feels like a unit, reliable like integration

17. 🏆🏆🏆
Feels like a unit, reliable like integration

18. Parallelization
●Surefire plugin
○ parallel=classes
○ forkCount=N
○ reuseForks=true
●Showcase
○ github.com/testcontainers/testcontainers-showcase
https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html

19. ●Using Docker-in-Docker (DinD)
●Testcontainers Cloud setup
●Parallelisation
https://www.atomicjar.com/2023/01/running-testcontainers-tests-on-gitlab-ci/

20. start.spring.io has it :)

21. Local development environment with SpringBoot 3.1
https://docs.spring.io/spring-boot/docs/3.1.0-SNAPSHOT/reference/htmlsingle/#features.testing.testcontainers.at-development-time

24. What’s next?
— github.com/testcontainers/testcontainers-java
— testcontainers.org
— slack.testcontainers.org