Lessons learned from Configuration Management. From Devop Oxford Meetup, February 2017

1. Mandi Walls | Technical Community Manager | mandi@chef.io
@lnxchk
Configuration
Management is
Old and Boring

2. Mandi Walls
Technical Community Manager for EMEA
@lnxchk
mandi@chef.io

3. Who you calling “old”?
If it’s so old, where’s the standard?

5. One-Minute History of CM
• There used to be a few expensive machines that did all the things
• The Operators configured them to meet specific needs of the programs
• There was a Cambrian-level explosion of technology
• Now, there are lots of cheap machines that do infinitely more things
• They still have to be configured to be useful
• Along the way some tools were written to help with this
• The End?

6. EVERY business is a software business
We’re going to be a software
company with airplanes.
– CIO, Alaska Airlines

7. Things Changed
• Technology is increasingly becoming a Core Competency for a lot of industries
• It’s hard to compete if you are still outsourcing all of your tech work
• Harder to set timelines
• More complicated to reprioritize
• Slower to respond to competitor’s advancements, feature requests, bug reports

8. Waterfall to Agile Era
• Great for software developers
• Ship it and Forget it
• Stranded operators at the end of a dev cycle or sprint
• Ignored the installation and running of software (still going on CD/DVD anyway)
• Represented a different kind of environment for CM
• Tipping Point – When do you need to buy CM from a vendor?
• Large estates still in the 100 – 500 host per-operator range (c 2006-2007)
• Dev Cycles still fairly slow, larger release cycles often quarterly or less but more agile bug
fixes, small features
• Operators wrote the tools they needed anyway because they had time
• Early vendors had much closer academic roots

9. Cloud Era
• While not everyone went to a “cloud”, the prevalence of VMs and hardware-on-
demand changed the need for Configuration Management
• In the same timeframe, Red Hat did things like getting government certified in
the US, tipping more Enterprise services onto Linux from commercial UNIX
• Installed estates evolved in complexity
• More virtual hosts on the hardware, whether own-site or in an MSP
• Divisions of applications toward 1-app-per-VM
• Specialization of machine types for different applications
• No additional investment in personnel in most places – economic downturn led to downsizing
• Beginning of the value proposition for investment in CM

10. Arbitrarily Declared DevOps/SRE Era
• Tool proliferation across all spectrums of the Operations lifecycle
• Open Source tools gain status, market
• Ecosystems expand for running and managing large estates, especially Linux
• Large-scale operations and system management gains voice as a practice apart
from “traditional” systems administration
• Configuration management is not only a required tool, but pushes traditional
Ops people to adopt more software dev-style practices
• Revision Control
• Infrastructure as Code
• Testing

11. Futurama
• Containers of some sort, running somewhere, doing something, being managed
by whatever
• Grab CPU only when you need it?
• Serverless? Unikernels?
• Who knows.
• Only some infinitesimally small amount of code is running here so far compared
to the amount of code in other environments
• For everyone else, pick one of the prior eras
• And they are all still in play

12. For the Time Being, We Still Care About CM
• Transitioning to newer practices requires stable foundations
• Building software faster means having environments that match
• Infringing a bit more on build and release cycles rather than staying in
Operationsland

13. Managing Cycle Environments
• The biggest complaint we hear from teams moving towards modern practices
• “None of our environments match”
• Production operations team doesn’t manage non-prod assets
• No sharing of resources, knowledge, tools hinders good code quality

14. Continuous Whatever
• Requires as much culture work as it does tools work
• Thinking about “always shipping” versus “periodic shipping”
• Automation of all parts of the pipeline
• Releases and release installs lose complexity and angst

15. Codified and Automated
• Migration away from GUI-based tools that hamper scaling, repeatability
• Treating the entire environment as a code base
• Risk reduction through smaller releases, traceable histories for all changes on
all platforms and envrionments

16. Container Environments
• Work required to build software onto containers is less about the containers
themselves and more about how developers think about the code that ships
• Build-once artifacts
• No retrieving of builds and re-pushing with the same build number, other bad
practices

17. “To the Left”
• More late-lifecycle struggles are being moved earlier in the cycle
• Security is the next big push
• No one likes to get to launch time and learn that Security isn’t going to let them
release their software onto the production networks, or that they built with the
wrong assumptions
• You can’t “bolt on” any number of issues at end of cycle – security, performance,
data privacy, UI

18. The game changer: rapid time to value
Innovation
Quality/
Complianc
e
Dynamic
Infrastructure
Infrastructure as Code
Automate the Stack
DevOps
+ +

19. CM is there to help go fast
• Build new hosts that meet your requirements
• Deploy code as often as necessary
• Make sure that monitoring, metrics collection, log management are in place
• Minimize time-to-market with automation

20. CM as a component of failure
• Reducing the investment in a new environment lowers the cost and risk
associated with experimentation
• Cloud definitely pushed the envelope on this aspect, but also brought complexity
that got managed by CM tools
• This is an interesting place that serverless tech will also enter for some projects

21. CM and LEAN
• Eliminate non-value-added action, reduce waste
• Pull over Push requires flexibility
• Kaizen - Continuous Improvement to all aspects
• Kaikaku - Disruptive Change when necessary
• Small Batch + Experimentation

22. CM and Containers
• Container platforms continue to evolve rapidly, requiring management of the
underlying layer to guarantee consistency
• Additional services for management, metrics, reporting to be maintained

23. Configuration Management is a key
component of IT Modernization
It has been entwined with Cloud, DevOps, and Continuous Delivery

24. Complexity is deceptive
The things we expect to mitigate complexity can instead just move it or
change its properties rather than eliminating it

25. Lessons from the Last Eras
• Automation
• Dynamic Infrastructure
• Infrastructure as Code
• Revision Control
• Automated Testing
• Collaboration
• CAMS and CALMS
• Embrace Failure
• Blameless Culture
• Focusing on Business Value

26. What Does it Mean for Chef?

27. Change in Product Lifecycles
• Old eras had longer enterprise buying cycles
• 5-year plans for IT products, large investment
• Even though technology executives tend to only stay in a position for a couple of
years
• Writing long-term plans with a short-term personal outlook
• Doesn’t reflect modern product cycles and technology ecosystem
• 5 years is an eternity in “internet years”

28. Skip It?
• Enterprise organizations coming out of their last buying cycle in the last couple
of years had no reason to evaluate large VM management systems, but
containers weren’t mature enough to fully migrate
• Anyone coming out of a buying cycle now might have last purchased IT in 2012
• They should have been looking at the cloud then, but were they? It was scary
• Did they follow up their last purchasing cycle with a skills update cycle?
If an IT organization buys something today on a 3- or 5-year
contract, what will they see next?

29. Damn, that’s a lot to think about
• It’s all tied together
• Software vendors sell to early adopters, mainstream adopters, and eventually to
doubters
• Configuration Management, for all of it’s “been around forever”-ness is still
selling to mainstream adopters
• The high-risk, conservative buyers, like insurance companies and utilities, still
lag
• Specialized equipment, like in telcos, or specialized regulation in insurance slow down adoption
• Shortage of available practitioners willing to work for those organizations slows
down adoption as well

30. THANKS!!

31. Join Our Upcoming Events
• Hands On Habitat London – March 8 at Skills Matter
• “Who Owns Open Source Security?” at Chef Users London – March 13 at HPE
• Save the Date – Chef Community Summit London – October 10 and 11
• https://chef.io
• Find us! https://events.chef.io/
• https://learn.chef.io