DOES SFO 2016 - Ross Clanton and Chivas Nambiar - DevOps at Verizon
1. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
DevOps @ Verizon
Ross Clanton, Fellow, DevOps
Chivas Nambiar, Director, DevOps and Public Cloud Platform Engineering
2. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
“Safe Harbor” Statement
NOTE: This presentation contains statements about expected future events and financial results that are forward-looking and subject
to risks and uncertainties. For those statements, we claim the protection of the safe harbor for forward-looking statements contained in
the Private Securities Litigation Reform Act of 1995. The following important factors could affect future results and could cause those
results to differ materially from those expressed in the forward-looking statements: adverse conditions in the U.S. and international
economies; competition in our markets; material adverse changes in labor matters, including labor negotiations, and any resulting
financial and/or operational impact; material changes in available technology; any disruption of our key suppliers’ provisioning of
products or services; significant increases in benefit plan costs or lower investment returns on plan assets; breaches of network or
information technology security, natural disasters or terrorist attacks or existing or future litigation and any resulting financial impact not
covered by insurance; technology substitution; an adverse change in the ratings afforded our debt securities by nationally accredited
ratings organizations or adverse conditions in the credit markets impacting the cost, including interest rates, and/or availability of
financing; any changes in the regulatory environments in which we operate, including any increase in restrictions on our ability to
operate our networks; the timing, scope and financial impact of our deployment of broadband technology; changes in our accounting
assumptions that regulatory agencies, including the SEC, may require or that result from changes in the accounting rules or their
application, which could result in an impact on earnings; our ability to complete acquisitions and dispositions; and the inability to
implement our business strategies.
Throughout this presentation, financial information shown excludes, where noted, non-operational or one-time items. As required by
SEC rules, we have provided a reconciliation of the non-GAAP financial measures included in this presentation to the most directly
comparable GAAP measures in materials on our website at www.verizon.com/investor.
3. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 3
• Serving 99% of the Fortune 500
• Serving 2,700 cities in 150+ countries
INTERACTIVE
ENTERTAINMENT
INTERNET
SERVICES
• Largest 4G LTE network
• 113.7M retail connections
INTERNET
OF THINGS
• Nation’s largest all-fiber network
• 4.7M video and 5.6M Internet customers
DIGITAL
MEDIA
Fortune 500 rank: #13
2015 capital investment: $17.8B
2015 revenues: $131.6B
GLOBAL
BROADBAND NETWORK
Who we are:
Delivering the promise of the digital world
4. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Evolving customer profile … and expectations
Digital – on any device and across devices.
Instant – all devices always aware anywhere.
Simple – intuitive to buy, integrate and consume.
Reliable – works all the time.
Personal – everything is predictive and for me.
5. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 5
Key Demand Drivers
4K TV (UHD)
Exponential
Virtual & Augmented Reality
Exponential
WIRELINE USAGE
PUBLIC
GLOBAL IP USAGE
55%Y/Y G% 49%
WIRELESS USAGE
67%Y/Y G % 61% 45%Y/Y G% 41%
Customer Behavior Driving Usage
Dec 13 Dec 14 Dec 15 Dec 13 Dec 14 Dec 15 Dec 13 Dec 14 Dec 15
Video Streaming
Incremental
Multiple Devices
Incremental
6. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Retooling the Consumer Business
6
Reduce business process complexity
• Rationalize the product set
• Business Process Re-Engineering
• Simplify Customer Experience Simple
Personal
Customer
Experience
Single
Profile
Single
Catalog
Single
Order
Single
Biller
Plan for the future
• Roadmap for 7-10 years
• Next Generation OSS/BSS systems
7. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Unified Customer Experience
COA
.COM
RETAI
N
SX
NSOP
NPD
SOP
BQTSDMP
NYNE
SOP
NOCV
SSP
SEFS CRISNBBE
V5 –
SCM
RIBS
2014
COA
.COM NSOPNOCV
SSP
SEFS CRISNBBE
VISION RIBS
• Improved common customer experience
• Foundation for IPTV platform
• Eliminate systems to improve quote to bill
accuracy
• Consolidate profiles across 9 systems to
improve response times and cross-channel
consistency
COA
.COM
SSP VISION
• Shutdown 140 systems to create national billing platform
• Enable seamless customer cross-regional moves
• Improved speed to market across all CMB products
• Simplified customer experience with single account ID
2015
2013
üEliminated vendor
costs
8. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Phase 1 – Systems Consolidation
8
Simplify customer experience
• Single view of the customer
• Context driven call-in experience
• Simplified billing experience
System wide Re-architecture
• Consolidation of data
• API gateway layers
• Simplified single user interface
• Common Analytics platform
Systems Consolidation
696 ➡ 229 (2011 – 2016)
9. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Phase 1 – Infrastructure Consolidation
9
Virtualization and Cloud
• 1000s of Physical servers eliminated
• 60%+ servers virtualized
Mainframe Consolidations
• Sites: 6 ➡ 4
• Mainframes: 39 ➡ 17
Consolidating Data Center Footprint
• Data Centers: 14 ➡ 7 (2011 – 2016)
20%
30% 40%
60%
85%
2011 2012 2013 2014 2015
Virtual Servers
Standalone Æ Virtualization Æ Cloud
0
5
10
2011 2012 2013 2014 2015 2016
Datacenters
Data Center Consolidation
10. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Phase 2 – DevOps as Acceleration Strategy
Our DevOps Aspiration
Consistently create and deploy
reliable software onto
scalable infrastructure enabling
immediate business and customer benefits
10
11. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
The Value Creation Loop – Desired Outcomes
11
Start with
an idea
Build minimum
viable product
Deploy and
scale
Get Customer
feedback
• Enable smart experiments
• Unleash developer creativity
• Build and scale seamlessly
• Code to customer ownership
12. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
The Value Creation Loop – Desired Outcomes
12
Start with
an idea
Build minimum
viable product
Deploy and
scale
Get Customer
feedback
• Enable smart experiments
• Unleash developer creativity
• Build and scale seamlessly
• Code to customer ownership
13. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Phase 2 – Enable and unleash
Standard Best in Class SDLC Platform
• Single Code Repository
• Single, open by default agile tracker
• Automated testing infrastructure
• Embedded Code quality and security scanning
Optimize for Enthusiasm Adoption
• Grassroots driven
• Business benefit success story showcases
• Six Sigma driven metrics validation
14. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 14
Widespread
Enterprise
Adoption
Proven
Ongoing
Benefits
Continuing
Platform
Enhancements 767/1087 Directional Applications On-boarded
ü Increased % of Functional test cases automated
and executed
ü Increased % of Application functions and
deployments performance tested
ü 30%+ Reduction in production IRs
ü 30%+ Y/Y reduction in production outages
ü # of Continuous Integration builds
(Build/Test/Deploy) on platform
ü % No touch deployments
ü Multiple daily & weekly deployments enabled
vs monthly changes
Quality
Velocity
Platform Adoption
CMB, 138
CAO, 30
DWP, 15
VES, 184
WLNO, 11CFO, 66
ITCPS, 33
PDMO, 9
VZW, 95
VZW-N, 79
ITSP, 8
NTS, 99
15. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
The Value Creation Loop – Desired Outcomes
15
Start with
an idea
Build minimum
viable product
Deploy and
scale
Get Customer
feedback
• Enable smart experiments
• Unleash developer creativity
• Build and scale seamlessly
• Code to customer ownership
16. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Build and Scale
Retool the private cloud infrastructure
• Self-service automation as core concept
• API driven infrastructure using Openstack + Cloudfoundry
• Bring brown-field infrastructure along
Enable public cloud usage
• Started with non-production workloads
• Retraining teams for infra-as-code
• Security and Compliance embedded
16
EZ
17. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Migration Journey
Lift &
Shift
Cloud
Native
17
Lift & Reinstall
• Build technical capability with cloud toolset
• Drive automation
• 30% infrastructure efficiency expected
Cloud Native Transition
• Refactor directional applications
• Enable innovation via scalable infrastructure
• multiple production versions, A/B experiments)
• Drive additional 40% efficiency
18. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
VZW.com
• Primary online customer destination
• Increasingly larger revenue growth
source
• Business and market drives constant
change
Benefits:
• Code ➡ Deploy multiple times a day
• E2E environments in minutes vs
weeks
• 70+% test case automation
CoFEE Anywhere(Customer care system)
• One stop shop for care
• Supports 40k+ agents and technicians
• Critical to customer experience
Benefits:
• Releases from monthly ➡ 2x/week
• Elastic horizontal scaling
• Cost of infrastructure reduced by 30%
• In transition to micro-service drven
architecture
18
Phase 2 – What success looks like for Systems
19. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
• Engineering Culture
• Open Source adoption and adaptation
• Enterprise wide backlog
• Demo driven transformation
• Gamification as enabler
• DevOps Cups driving adoption
• Cash for success
• Agile transformation across multiple lines of business
• Open backlogs
• Knowledge and skills sharing
19
Phase 2 – What success looks like for teams & culture
• Optimize for enthusiasm
• One size does not fit all
• Constant improvement
becomes second nature
• Take the first step!
20. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Value Creation Enablers – Where we are today
20
ü Best in Class Agile SDLC
• Scalable Cloud Infrastructure
• Great Developer Experience
• DevSecOps enabled organization
21. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
The Easy Part was
done…
So for Phase 3 we
brought in:
Ross Clanton!
21
Dev
Ops
Been there, done that, and I have a T-
Shirt to prove it
-Ross Clanton, 2016
22. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Where next?
Accelerating Innovation!
22
Dev
Ops
23. Scaling Engineering Culture & Practices
23
Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any
unauthorized persons or third parties except by written agreement.
Our goal is to grow and scale our Engineering capabilities, with deeper
competencies in Agile, DevOps, Cloud Native architecture, and security by
design.
24. How We Accelerate: Optimize for Agility
24
Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any
unauthorized persons or third parties except by written agreement.
25. How We Scale
Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any
unauthorized persons or third parties except by written agreement.
Embed
• Dedicated Coaching
• Co-creation
Enable
• Immersive Experiences (i.e. Dojos)
• Workshops
Establish
• Patterns, Practices, & Frameworks
• Code Examples / Reference Apps
• Self-Service Tooling & Training
• Communities of Practice
BaseService
Premium
Service
White-Glove
Service
26. Build Dojo Locations
Dojos are trending in the industry as an effective way to transform skills, incubate innovation,
and foster engineering culture
26
Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any
unauthorized persons or third parties except by written agreement.
Open Innovation Labs“Target rebuilds it’s Engineering
Culture, Moves to DevOps”
“CompoZed Labs aims to allow
parts of Allstate acting more like a
startup”
“Utilizing distributed dojo’s to
transform our workforce”
Vendor Labs ModelInternal Dojos
27. Engineering Practices
Operationalizing Dojos
27
Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any
unauthorized persons or third parties except by written agreement.
Roles
Optimize Dojo LocationsEstablish Engineering Practices Organization
Dojo Manager
Dojo Ops
DevOps Coach
Cloud Coach
App Sec Coach
Operations Strategy
• Dojo locations established at key technology
locations
• Regional approach provides hubs to travel to
• 40-50 seats per Dojo
• Staffing per location
• Scale space and coaches per demand
Community Engagement
Mgr
Agile
Engineering
DevOps
Cloud Native
Public Cloud
API / Services
Application
Security
28. How We’ll Know We’re Successful: Measuring Progress
Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any
unauthorized persons or third parties except by written agreement.
Are we getting better? (Capability Measures)
+
Are we seeing results? (Outcome Measures)
Community Challenge: Build a widget in Hygieia to integrate DORA results
29. Where do we need help
29
Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any
unauthorized persons or third parties except by written agreement.
How to move to a more customer aligned product and service
based organization…..
30. Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or
distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Thank you.
30