A company attempted a large-scale DevOps initiative to improve their development and production processes, but faced many challenges that ultimately led to failures in production. The company cultures and teams were spread across geographies and silos, leading to tension when a new elite team was brought in. Technical debt from 10 environments and long stage gates caused knowledge vacuums and splits between new and experienced employees. Key lessons included needing more empathy, data-driven decisions, treating DevOps as an organizational change, and ensuring production-like environments in development to prevent failures in production.

1. GOING BIG AND F*#&#*G UP
A Cautionary Tale
@mikebaukes
1

2. THE BUSINESS
s
p d
O e
v Disruption
MarketWide ov
e r
D p
p
A
A History of Success
A New Market in 9 months
2

3. THE PROJECT
Funding a DevOps initiative using "risk”
y
t
i
r s
What do we have? cu p
e O
S ev
/ of production?
k
Is it representative D
is e
R v
lo
Are we at risk?
3

4. COMPANY CULTURE
Teams Spread across Geographies and Organization Charts
Average Request fulfillment was around 2 weeks
Rife with process debt
4

5. TEAM CULTURE
We recruited the best
Immediately Caused Tension
Elitism & Arrogance began to grow
5

6. THE PROCESSES
Risk = MS/DE Sign-offs
Project Managers = Test Results/Environment Documentation
SDLC = VC Inf/Repeatable Env
ITIL = DSL/DML/CMDB & ST
Architecture = TRM/Enterprise Continuum
Automation using Chef
6

7. TECHNICAL DEBT
10 Environments, Multiple Stage Gates (2 weeks)
90% on Writing Code, 10% on education
Split in 2 groups (newbies/old-school)
Knowledge Vacuum
7

8. PROGRESSION/REGRESSION
8

9. LESSONS
1. Empathy
2. Get the Data!
3. DevOps is an HCM issue WTF?
4. In production and not in dev means #fail
9

10. @mikebaukes
THANKS
@scriptrock
@devops
10