Configuration Management is at the core of Ops. It’s the biggest enabler of any compute operation, small and big. In the past decade, we have switched from thinking about the machines we are configuring, to think about the software and services we are controlling. With that change of mindset, so did the tools we are using. Traditional tools like Puppet, chef, salt and Ansible are slowly declining while new tools such as Terraform, Pulumi, Helm and Kustomize are on the rise. In this talk I will try to describe the pain-points and the opportunities of this transformation as well as suggesting a future direction based on tools developed at the big-tech companies (Mainly facebook and google).

1. Configuration Management
In the cloud-native era

2. Shahar Mintz
Married to Rachel
Father for Ariel and Yama
DevOps Consultant/Freelance
Previously @ Wix, Packet,
Facebook, Onavo, Red Hat,
Qumranet

3. 2013 -> 2018
2013 - Onavo
~400 physical servers
Puppet in charge of:
- OS configuration (sysctl,
services, files etc)
- Monitoring
- Machine Access
- Firewall configuration
- Service Discovery
2018 - Packet
~ 30 installations
- OS Config:
- Salt
- Provisioning:
- Terraform
- Monitoring:
- Prometheus & Grafana (Manual)
- Service Discovery:
- Consul
- Workload Management:
- Nomad

4. Why configuration
management is
important?

7. A brief history of
configuration
management

8. Imperative vs Declarative

9. 1st gen: CFEngine - 1993
State:
Server management done by
scripts.
Challenges:
Different OS flavours
Machines in different states

10. CFEngine
Solution:
Declarative DSL.
Focus on Promises vs
Obliges.
bundle agent example {
files: "/tmp/testfile"
create => "true",
edit_line => proper_greetings;
}
bundle edit_line proper_greetings {
delete_lines: ".*";
insert_lines: "Hello World!";
}

11. 2000s - Dot com boom

12. CFEngine - The bad things
Written in C - Hard to
extend.
Limited file content
operations.
bundle agent example {
files: "/tmp/testfile"
create => "true",
edit_line => proper_greetings;
}
bundle edit_line proper_greetings {
delete_lines: ".*";
insert_lines: "Hello World!";
}

13. 2nd Generation
Puppet
- 1st Release: 2004
- DSL: Proprietary
- File content: ERB
Chef
- 1st Release: 2009 (was used
before)
- DSL: Ruby
- File content: ERB

14. Model -> Controller -> View
Model Controller View
Chef attributes Recipes (ruby) ERB
Puppet variables modules ERB

15. 2nd Generation
- Better server architecture
- Database backend
- Machine attribute/variables (facter/ohai)

16. 2010s - The Cloud

17. Chef & Puppet disadvantages
- No parallelism
- Long execution times

18. 3rd Gen
- Push based execution
- Focus on remote parallel
execution
- Asset management
Salt:
- First release: 2011
Ansible:
- First release: 2012

19. Containers & Cloud Native applications

20. The Cloud Native Era

21. Cloud Native Infrastructure
Provisioning and SaaS/IaaS:
- Terraform
- Pulumi
- CloudFormation
Workload Management
- Kubernetes
- Helm
- Kustomize
- Nomad
Monitoring:
- Prometheus
- InfluxDB
- Grafana
Networking/Routing:
- Envoy
- Linkerd

22. HELM AND KUSTOMIZE

23. Recap

24. Configuration Management challenges
1. Configuration evaluated on the production machines
2. Hard to test (result of problem #1)
3. Too many configuration formats
4. YAML+Templating = 💔

25. Bridging the gap

26. Bridging the gap

27. Software wants:
Static Data (i.e JSON)
{
"person1": {
"name": "Alice",
"welcome": "Hello Alice!"
},
"person2": {
"name": "Bob",
"welcome": "Hello Bob!"
}
}
Software vs. Human
Human writes:
Python
def person(name='Alice'):
return {
“name”: name,
“welcome”: 'Hello %s!' % name,
}
def main():
return {
“person1”: person(),
“person2”: person('Bob'),
}

28. “Because our systems are ultimately managed by humans, humans
are responsible for configuration. The quality of the human-
computer interface of a system’s configuration impacts an
organization’s ability to run that system reliably.”
- Štěpán Davidovič, Google SRE Workbook
https://sre.google/workbook/configuration-design/

31. Protoconf Goals
- Deliver configs to all clusters in seconds,
not minutes.
- Configs should have schemas with type
safety
- Configs should be coded, then materialized
- Config changes should be reviewed
- Configs should be easy to test and validate
- Configs could be consumed by all popular
languages.
- Both humans and machines should be able to
change configs

32. Define the config schema
The developer will define
the config struct in
protobuf
// file: ./src/myproject/myconfig.proto
syntax = "proto3";
message MyConfig {
uint32 connection_timeout = 1;
uint32 max_retries = 2;
NestedStruct another_struct = 3;
}
message NestedStruct {
string hello_world = 1;
}
https://developers.google.com/protocol-buffers
https://docs.protoconf.sh/getting-started/

33. Add validations
"""
file: ./src/myproject/myconfig.proto-validator
"""
load("myconfig.proto", "MyConfig")
def validate_connection_timeout(config):
if config.connection_timeout <= 3:
fail("connection_timeout must be 3 or higher, got: %d" %
config.connection_timeout)
add_validator(MyConfig, validate_connection_timeout)
https://docs.protoconf.sh/getting-started/

34. Code your config
The developer will then
create a `.pconf` file to
populate the config struct
with the required values.
"""
file: ./src/myproject/myconfig.pconf
"""
load("myconfig.proto", "MyConfig", "NestedStruct")
def main():
return MyConfig(
connection_timeout=5,
max_retries=5,
another_struct=NestedStruct(
hello_world="Hello World!"
)
)
https://docs.protoconf.sh/getting-started/

35. Compile
$ protoconf compile .
{
"protoFile": "myproject/myconfig.proto",
"value": {
"@type": "type.googleapis.com/MyConfig",
"connectionTimeout": 5,
"maxRetries": 5,
"anotherStruct": {
"helloWorld": "Hello World!"
}
}
}
https://docs.protoconf.sh/getting-started/

36. Consume
channel = grpc.insecure_channel("localhost:4300")
stub = ProtoconfServiceStub(channel)
config = MyConfig()
for update in stub.SubscribeForConfig(
ConfigSubscriptionRequest(path="myproject/myconfig")
):
update.value.Unpack(config)
print(config)
https://www.grpc.io
https://docs.protoconf.sh/getting-started/

37. After Merge

38. Mutation RPC
https://docs.protoconf.sh/mutation-rpc/

39. Learn More & Contribute
- Docs site:
- https://docs.protoconf.sh
- Star us on Github
- protoconf/protoconf
- Join us on Discord
- https://discord.protoconf.sh
- Follow us on Twitter:
- @protoconfdev