1.
Top Lessons Learned While
Researching and Writing The
DevOps Handbook

2.
‘tis the season for town hall debate formats 
Mark Tomlinson
Performance Sherpa
@mark_on_task
Andi Grabner
Performance Advocate
@grabnerandi
Gene Kim, CTO
Researcher and Author
@RealGeneKim

3.
Gene, thoughts on companies that move
to more rapid deployment mode and
their success?

4.
The DevOps Handbook Is Here!
•5+ Years & Now Avail!
•23 Chapters
•48 Case Studies
•48 Images
 192 footnotes
 503 endnotes
 98,124 words

5.
High Performers Are More Agile
200x 2,555x
more frequent
deployments
faster lead times
than their peers
Source: Puppet Labs 2015 State Of DevOps Report: https://puppet.com/resources/white-paper/2016-state-of-devops-report

6.
High Performers Are More Reliable
3x 24x
lower change
failure rate
faster meantime to
recover (MTTR)
Source: Puppet Labs 2015 State Of DevOps Report: https://puppet.com/resources/white-paper/2016-state-of-devops-report

7.
High Performers Are More Secure and Controlled
2x 29%
less time spent
remediating
security issues
more time spent
on new work
Source: Puppet Labs 2015 State Of DevOps Report: https://puppet.com/resources/white-paper/2016-state-of-devops-report

8.
High Performers Win in the
Marketplace
2x 50%more likely to
exceed profitability,
market share &
productivity goals
higher market
capitalization growth
over 3 years*
Source: Puppet Labs 2015 State Of DevOps Report: https://puppet.com/resources/white-paper/2016-state-of-devops-report

9.
@mark_on_task
What is the business value of
adopting DevOps?

10.
@RealGeneKim
Increased Flow of High Quality Value
Test Driven Development
Automated Deployments
Infrastructure as Code
Virtual/Cloud/PaaS/Containers
Break the Monolith to
Reduce Code & Dependency
Complexity
Shift-Left Performance
Automated Feedback Loops
@grabnerandi

11.
@RealGeneKim
Real User Feedback: Building the RIGHT thing RIGHT!
Experiment &
innovate on
new ideas
Optimizing what is
not perfect
Removing
what
nobody
needs
@grabnerandi

12.
@RealGeneKim
Faster Lead Times to User Value!
@grabnerandi

13.
@RealGeneKim
24 “Features in a Box” Ship the whole box!
Very late feedback 
@grabnerandi

14.
@RealGeneKim
Continuous Innovation and Optimization
1 “Feature at a Time”
Optimize Before DeployImmediate Customer Feedback
@grabnerandi

15.
What are your favorite DevOps
transformation stories, top picks?

16.
@RealGeneKim
CASESTUDY
2005
Google Web Services
Transformation

17.
@RealGeneKim
CASESTUDY
The Birth and Death of
Etsy Sprouter

18.
@mark_on_task
Andi, how was DevOps used to
transform Dynatrace?

19.
@RealGeneKim
CASESTUDY
Leading by Example
Using DevOps to remain the
Leader in a disrupting market
@grabnerandi

20.
2011: APM about to be disrupted!
• Migrate from On-Prem to VM, Cloud, Containers and PaaS
• Architectures include micro-services, on-demand scaling, self-
healing
• ”Cloud Natives“ demand SaaS based solutions
• Digital Transformers demand Analytics for Biz, Dev, Ops & Sec
• Many new players on the market
@grabnerandi

21.
Challenges to master!
• Bridging the gap between ”New Stack“ and “Enterprise Stack“
• Deploying the same way our customers do: Continuously!
• Not disrupting current operations and slower moving customers
• Aligning 300+ engineers across 3 different geos
@grabnerandi

22.
Believe in the mission impossible
6months
major/minor release
+ intermediate updates
+ weeks to months
rollout delay
sprint releases
(continuous-delivery)
1h: code to production
@grabnerandi

23.
2 major releases/year
customers deploy & operate
on-prem
26 major
releases/year, 170
prod deployments /
day, self-service
online sales , SaaS &
Managed
2011 2016
@grabnerandi

24.
#1: Going from 6 Months to 1 Month On Premise Updates
• Challenge: Monolith download too big for our customers
• Impact: Update Process was error prone and “All or Nothing“
• Solution: Componentize, Automate Rollout/Rollback Capability,
A/B Rollout Model
Increased velocity uncovered bottlenecks!
@grabnerandi

25.
#2: Education on Frequent Updates
• Challenge: Release Education used to happen 60-90 Days
after the release
• Impact: Upgrade to latest version happened very late
• Solution: Education Integrated into Continuous Delivery:
Dev Blogs, YouTube Videos...
Increased velocity uncovered bottlenecks!
@grabnerandi

26.
#3: Availabilty of Development / Test Environments
• Challenge: Supporting many different tech stack makes it hard
to maintain it
• Impact: Long running support tickets and long feature
development
• Solution: Infrastructure as Code gives “On Demand“ access to
these enviornments
Increased velocity uncovered bottlenecks!
@grabnerandi

27.
• Sprint Reviews Done on “dynaSprint“
• Daily Builds get deployed on “dynaDay“. Sprint builds to “dynaSprint“
• If you can only show it “on your dev machine“ its NOT DONE!
• Deploy Sprint Builds into our internal Production Enviornment
• We monitor Website, Support, Licensing, Community ... With Dynatrace
• If we break our own backoffice software we ALL feel the pain right away
Confidential, Dynatrace, LLC
Tip #1: Increasing Sprint Quality
@grabnerandi

28.
Confidential, Dynatrace, LLC
Tip #2: Raising Awareness of Pipeline Quality
https://github.com/Dynatrace/ufo @grabnerandi

29.
• Which Features to Optimize? Which Features to „Phase Out“
• Allows Reducing Technical and Business Debt
Confidential, Dynatrace, LLC
Tip #3: Feedback Loops in SaaS & On-Premise
@grabnerandi

30.
@mark_on_task
What are other DevOps
learnings in this new chapter of
the DevOps adventure?

31.
DevOps Enterprise: Lessons Learned
• On Oct 19-21, we held the second DevOps Enterprise Summit, a
conference for horses, by horses
• Speakers included fifty leaders from:
• Macy’s, Disney, Target, GE Capital, Western Union, Sherwin Williams, Blackboard,
Nordstrom, Telstra, US Department of Homeland Security, CSG, Raytheon, IBM,
Ticketmaster, MITRE, Marks and Spencer, Barclays Capital, Microsoft, Nationwide
Insurance, Capital One, Gov.UK, Fidelity, Rally Software, Neustar, Walmart, PNC,
ADP, …

32.
Observations
• They were using the same technical practices and getting the same sort of
metrics as the unicorns
• Target: 100+ deploys per week, < 10 incidents per month, enabled 53 business
initiatives
• Capital One: 100s of deploys per day, lead time of minutes
• Macy’s: 1,500 manual tests every 10 days, now 100Ks automated tests run daily
• Disney: Has embedded nearly 100 Ops engineers into LOB teams across the
enterprise
• Nationwide Insurance: Retirement Plans app (COBOL on mainframe)
• Raytheon: testing and certification from months to a day
• US CIS: security and compliance testing run every code commit

33.
33

34.
@RealGeneKim
DevOps enables dynamic,
learning organizations

35.
Dr. Steven Spear
“While designing
perfectly safe systems is
likely beyond our abilities,
safe systems are close to
achievable” when the
four following conditions
are met…”
Source: Dr. Steven Spear

36.
@RealGeneKim
Dr. Steven Spear’s Four Capabilities
1. See problems as they occur
2. Swarm and solve problems to create new
knowledge
3. Spread new knowledge throughout the
organization
4. Leaders create new leaders
Source: Dr. Steven Spear

37.
@RealGeneKim
Open / sharing cross team culture
 e.g: Capital One
open sourcing
their Hygiea
Dashboards
 Larger Enterprises
sharing their
stories, pitfalls and
lessons learned
via Blog or
Conferences (such
as DOES, DOD,
Meetups, etc.)
Tip: Start internal DevOpsDays to promote
lessons learned within the organization
@grabnerandi

38.
@mark_on_task
Mark, lead conclusions, wrap
and move to Q & A

39.
@RealGeneKim
Q & A
Mark Tomlinson
Performance Sherpa
@mark_on_task
Andi Grabner
Performance Advocate
@grabnerandi
Gene Kim, CTO
Researcher and Author
@RealGeneKim

40.
Lots of cool stuff!
Join us, stay connected, learn more!
Order the “DevOps
Handbook” now on Amazon,
click HERE
The “DevOps Handbook”
catalogues “high performing
DevOps organizations” and
what they have in common.
It then provide prescriptive
guidance so that other
organizations can replicate
their results.
Follow the Dynatrace Devops
story, join upcoming Webinar
“Lessons Learned Moving
Dynatrace from an On-Prem
Company to One that is
Cloud Native” click HERE

41.
Lots of cool stuff!
Join us, stay connected, learn more!
Learn more about the
DevOps Enterprise Summit,
click HERE
The goal of this event is to
give leaders the tools and
practices they need to
develop and deploy software
faster and to win in the
marketplace.
Join our PurePerformance
Podcast, click HERE
Gene Kim will join us live to
take us thorugh more details
of top insights discovered
while co-authoring The
DevOps Handbook

42.
Performance management for the digital customer age