DevOps is everywhere, but too often, people think they can buy “DevOps in a box” and just sprinkle some tools and automation over your broken or slow (or even super-fast AWS) stack. But we all know that software delivery is still hard. So what is this crazy DevOps thing, and why and how does it make things better? In this session, Jez and Nicole talk about what they’ve found working with dozens of organizations and conducting the largest DevOps research studies to date, covering over 23,000 data points across 2,000 organizations around the world. We start with the outcomes that companies care about: organizational performance, software delivery performance, and software quality. We then define what DevOps is, how you measure it, and how the best, most innovative teams and organizations are using it to drive improvements in performance and quality.

1. AWS re:INVENT
Tools Won’t Fix Your Broken DevOps
N i c o l e F o r s g r e n , P h D @ n i c o l e f v
J e z H u m b l e @ j e z h u m b l e
© 2 0 1 7 D e v O p s R e s e a r c h a n d A s s e s s m e n t L L C . L i c e n s e d C C - B Y - S A .
N o v e m b e r 2 8 , 2 0 1 7

2. Outline
What is DevOps?
Why should I care?
How does it work?
How do I improve performance and quality?
What is DevOps culture?

3. water

4. water
scrum

5. water
scrum
fall

6. Jon Jenkins, “Velocity Culture, The Unmet Challenge in Ops” | bit.ly/1vJo1Ya

7. The DevOps Movement
A cross-functional community of practice dedicated to the study of
building, evolving, and operating rapidly changing, secure, resilient
systems at scale.

8. IT Performance Matters!
“Firms with high-performing IT organizations were twice as likely to
exceed their profitability, market share and productivity goals.”
bit.ly/2014-devops-report/ bit.ly/2015-devops-report/ bit.ly/2016-devops-report/ bit.ly/2017-devops-report/

9. For Nonprofits, Too
High performers were also twice as likely to exceed objectives in:
• Quantity of goods and services
• Operating efficiency
• Customer satisfaction
• Quality of products or services
• Achieving organization or mission goals.

10. IT/Software Delivery Performance
Lead time for changes
Release frequency
Time to restore service
Change fail rate

12. Continuous Delivery

13. Quality

14. Change Management
All production changes must be approved by an external body (e.g.,
change approval board, manager, etc.) before deployment or
implementation (R)
We have no change approval process
We rely on peer review to manage changes

15. Now for Management Stuff
We all know managing WIP is important, right?

16. Now for Management Stuff
We all know managing WIP is important, right?
Prediction from WIP to ITPerf isn’t very strong

17. Now for Management Stuff
We all know managing WIP is important, right?
Prediction from WIP to ITPerf isn’t very strong
What’s going on?

18. Lean Management SEM

19. Product Management SEM
• Working in small batches and use of MVPs
• Visibility into the flow of work across the value stream
• Seeking—and acting on—feedback from customers
• Giving teams the authority to create and change specs

20. Product Management SEM
Evaluating well-designed and executed experiments that were
designed to improve a key metric, only about 1/3 were successful
at improving the key metric!
- Online Experimentation at Microsoft, Kohavi et al, stanford.io/130uW6X

22. Product Management SEM
"I think building this culture is the key to innovation. Creativity
must flow from everywhere. Whether you are a summer intern or
the CTO, any good idea must be able to seek an objective test,
preferably a test that exposes the idea to real customers.
Everyone must be able to experiment, learn, and iterate. Position,
obedience, and tradition should hold no power. For innovation to
flourish, measurement must rule.”
- Greg Linden glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html

23. High-trust Culture
How Organizations Process Information
Westrum, “A Typology of Organizational Cultures” | bmj.co/1BRGh5q

24. Likert-type Scale

25. Effective Teams

26. Dealing with Failure
In a complex, adaptive system, failure is inevitable.
When accidents happen, human error is the starting point of a
blameless post-mortem.
Ask: How can we get people better information?
Ask: How can we detect and limit failure modes?

28. Disaster Recovery Testing
“For DiRT-style events to be successful, an organization first needs
to accept system and process failures as a means of learning. … We
design tests that require engineers from several groups who might
not normally work together to interact with each other. That way,
should a real large-scale disaster ever strike, these people will
already have strong working relationships”
- Kripa Krishnan, Director, Cloud Operations, Google
Kripa Krishnan | queue.acm.org/detail.cfm?id=2371297

29. How to Impact and Influence Culture
“What changed the culture was
giving employees the means by
which they could successfully do
their jobs. It was communicating
clearly to employees what their
jobs were and providing the
training and tools to enable
them to perform those jobs
successfully.”
- John Shook
sloanreview.mit.edu/article/how-to-change-a-culture-lessons-from-nummi/

30. Conclusions
We CAN have it all, or at least tempo AND stability.
DevOps culture and practices have a measurable impact on IT & org
perf and quality.
Culture can be measured and changed.
Technology and agility do matter—but it’s not enough.

31. Questions
● New book coming out February 2018
● Our research: https://devops-research.com/research.html
● Assess your organization: https://devops-research.com/

32. Thank you!