1. Safeguarding - protecting your clusters
from misbehaving clients.
Tom Scott

2. ● Principal Engineer - Conduktor
Gateway
● 6 years in Kafka across many roles
● 15+ yrs in all things data
Who am I?

3. ● To minimise outages
● To respond gracefully to change
● To reduce friction between domains
Result: Velocity!
Start with Why

4. The practice of protecting your data and
infrastructure from intentional and
unintentional threats, both internal and
external.
What is Safeguarding?
Data Governance
“
Technical Best
Practices
Safeguarding

5. A collection of loose, negotiable contracts enforced by
humans.
Case in point: The “production readiness” check
These suffer from:
● Relevance - scoped for multiple applications in multiple technologies
● Timeliness - Seen as a box check to be completed just before go live
● Perception - Rules to be bent/ignored, “what can we get away with?”
Stage 1: safeguarding by habit

6. “Your application is behaving badly”
Now what?
● Uncomfortable decisions
● Temporary solutions (that become permanent)
● Tech debt
Stage 2: reactive automation

7. ● Enforced at the earliest point
● Applied in every environment
● Specific and detailed information on breaches
Examples include:
● Schema validation
● Partition count limits
● Automatic encryption
Stage 3: proactive automation

8. How? (but actually where)
A Client Plugin
A Broker Plugin
A Proxy

9. Application
Proxy
Application
Encryption
Interceptor
Schema Validation
Interceptor
Ring fenced
Conduktor Gateway - a Kafka
Proxy
● Interceptors - functions that perform
transformation/filtering/projection on Kafka
requests/responses
● Gateway embeds one or more interceptors
● To a client:
Gateway == Kafka Brokers

10. ● Proactive safeguarding:
○ Prevents outages
○ Resets perception
● Enforced inter-domain contracts help:
○ React gracefully to change
○ Reach broader technical goals
○ Reduce friction
● Proactive safeguarding does not:
○ Define governance/best practice policies
○ Fully replace reactive solutions
In summary

11. Q&A