IBM InterConnect 2016 Greg Hodgkinson 2238 Thriving DevOps at BMI (Prolifics)

CONNECT WITH US:
IT: Customized to Your Advantage
Thriving DevOps at Broadcast
Music Inc. DOD-2238
Growing from nothing to world class.
Public | Copyright © 2014 Prolifics
Greg Hodgkinson – Prolifics, Global
Director of Lifecycle Tools and Methodology
Jim Harvey - Broadcast Music Inc., Sr. Dir.
of Quality Assurance and Release
Management

CONNECT WITH US:
What attendees will learn
 What has made DevOps thrive at Broadcast Music
Inc.?
 What are the conditions and driving forces behind a
thriving DevOps practice?
 What does this thriving practice look like?
 Who is involved, what tools are they using, how have
they benefitting from these tools?
 What are they achieving?
 How is DevOps making a difference to the business?
2

CONNECT WITH US:
Introductions

CONNECT WITH US:
Who Are We Making Happy?
4
Everyone!
(that listens to music)

CONNECT WITH US:
Speaker Introduction
5
JIM HARVEY
Senior Director of Quality Assurance and Release Management
Broadcast Music Inc.
Jim Harvey is the Senior Director of Quality Assurance and Release
Management for Broadcast Music Inc. (BMI).
He has over 16 years’ experience in Software Development. Jim is currently
evolving the Software Development Life Cycle
at BMI as it relates to Requirements Management, Software Testing and Change
Control management.
GREG HODGKINSON
Director of Lifecycle Tools and Methodology
Prolifics
Greg has 20 years of experience in assisting customers achieve success by
optimizing their teams. Optimizations include adopting agile practices, DevOps
tools and practices, and SOA techniques.
He is very much a practitioner with responsibility for successful delivery at a
number of leading companies. He presents on DevOps and agile SOA
approaches at major IBM events, has co-authored a Redbook on SOA solutions,
and contributes to DeveloperWorks.

CONNECT WITH US:
Broadcast Music Inc. at a Glance
 Broadcast Music, Inc. - 1939
 Performing Rights Organization
(PRO) - Pay public performance
royalties
 Operate on a non-profit-making
basis
 7 locations: Nashville, New York,
Los Angeles, Atlanta, Miami, Puerto
Rico, London
 600 employees, 7.5 million works,
over 500,000 songwriters and
composers
6

CONNECT WITH US:
The Performers
7
Israel Kamakawiwoʻole
Louis Armstrong
Judy Garland

CONNECT WITH US:
The Writers
8
“What a wonderful world” George David
Weiss
Bob
Thiele
“Somewhere over the rainbow”
1939
1967

CONNECT WITH US:
2016: The Digital Music Ecosystem
 Over 500 digital music
services worldwide
 Offer consumers the
opportunity to legally access
up to 26 million songs
 Challenge: Stay relevant
and competitive?
9

CONNECT WITH US:
Prolifics at a Glance
10
5-Years Compound
Annual Growth Rate
19%
Employees
Worldwide
1,500
Global Presence
United States, United Kingdom, Germany, India
20+Technology
Accelerators
550+
Technical
Certifications
Over 10 Technology and Solutions
Awards since 2009 including
Business Agility, Customer Integration
and Digital Experience, the first ever
Beacon Laureate for Business Agility
Over 160 global customers
are currently Fortune
1000 companies
Best-in-class architects and
specialty experts:
BPM, Integration, Digital Experience,
Security, Testing, Business Analytics
and Enterprise Content Management
End-to-End Project Expertise
Rate of Repeat Engagements*
91%
Years in
Business
35+
Offices
14
Awards
Technology Expertise
Fortune 1000
*based on % revenue Source – December 2013 internal revenue metrics

CONNECT WITH US:
Challenge and
Success
11/14/2016 11

CONNECT WITH US:
Why the need for DevOps at BMI?
 Rapidly innovating music
industry
 Means BMI needs rapid
innovation themselves
 New music services => new
integrations
 New offerings = > new
systems
 Explosion of artists =>
increased need for scale
==========================
=
Net: Need to deliver a lot of
software innovation, quickly.
12

CONNECT WITH US:
The BMI Transformation
 Over a period of 4-5 years BMI have fundamentally changed
their ability to deliver software innovation.
13
New Approaches New Runtime Platforms New Development Tools
Agile
BPM
SOA
DevOps
IBM BPM
IBM WebSphere Portal
IBM ODM
WebSphere DataPower
IBM Integration Bus
WebSphere ESB
IBM CastIron
IBM DataStage
Oracle
IBM WSRR
Focal Point
DOORS NG
Rational Software Arch.
Rational Team Concert
Rational Quality Mgr.
IBM UrbanCode Deploy

CONNECT WITH US:
How Did They Change (Successfully)
So Fast?
 They key to this rapid transformation: Good processes and
tools!
14
To be successful with these…
…required that they be successful with these.
In a nutshell: all the things that helped them be efficient!
DevOps especially relevant.

CONNECT WITH US:
Adding Agile and RTC
 First challenge: Herding the
cats!
 Scrum
 Structure for teams
 Focus of sprints
 Course correction
 Simple – get going quickly!
 RTC
 Progress visible
 Keeps everyone on same page
 Easy configuration
management
 Opportunity for integrated
builds
15

CONNECT WITH US:
Toe in the Water: Continuous Integration
 Opportunity: Integrated
builds for quality feedback
loop
 Continuous integration of
WESB apps
 Java apps easy, start tackling
newly adopted WESB
 Developed build machine
plugin to automate builds of
WESB ears
 Used in CI cycles: immediate
feedback for devs (learning
new tech)
 Little effort -> lots of value
16

CONNECT WITH US:
Obvious Next Step: Automated Deploys
 Challenge: Delays to get
changes into test
 Continuous deployment of
WESB apps
 Added automated deployment
of WESB ears to plugin
 Either on demand, or as soon
as ready
 No more waiting; testers
testing earlier
 Reduced delays mean more
testing done; quality increases
17

CONNECT WITH US:
Expand Across Enterprise Architecture
 Opportunity: Success
breeds success
 Full-stack build and deploy
 Made use of Prolifics
investment
 Common framework
 Leveraged common steps
 Already had plugins for IIB
and Portal
 Expanded to all app types
 Integrated stack no longer held
back by slowest individual
release pipeline
18

CONNECT WITH US:
Added Database Deployments
 Challenge: But what about
the data?
 New plugin for automating
deploy of schema updates
 Implemented using same
framework and standards as
apps for simplicity
 DBAs place scripts into source
control
 Automation picks up and
upgrades schema
 Works across all environments
19

CONNECT WITH US:
Added Publishing of Services
 Challenge: And what about
the service registry?
 New plugin to publish to
WSRR
 Again, same framework and
standards leveraged for
simplicity
 Uploads WSDLs, XSDs, and
other file changes
 Updates version and other
metadata
 Consistent across all
environments
 One less thing to remember to
do
20
*Bad luck Brian

CONNECT WITH US:
Ramping up on Automated Deployment
 Challenge: Push button
deployments – too many buttons!
 Purpose-built deployment
engine: UrbanCode Deploy
 Wrap existing deployment
automation as UCD plugins -
quickly and painless move
 Orchestration
 Cross-tier deployments simple
 Easier to elaborate deploys of
individual components too
 Snapshots of complex apps
simple
 Conditional gates on
environments
21

CONNECT WITH US:
Integrated Automated Testing
 Opportunity: Include existing
automated tests
 Plugin to wrap testing tool(s)
 Tests moved into source control
 Builds pull out tests and execute
them automatically
 Can then
 Trigger when you need,
 Schedule,
 Or include as part of
build/deploy/test process
 Lots more value from test scripts
 Regression testing
 Smoke testing
22

CONNECT WITH US:
Dockerising Build Agents
 Challenge: Build
infrastructure inflexible time
sink
 Created Docker images of
all build agent
configurations
 Install build agent software
 Install automation
 Install supporting software – in
some cases this is
considerable
 Easier to isolate build agents
 Easier to spin up new agents
 Easier to refactor infrastructure
23

CONNECT WITH US:
Next: Automated Provisioning of
Environments
 Challenge: Full set of test
environments takes up a lot
of tin
 Create Docker images for
all test environment
configurations
 Saves time – also allows new
behaviors
 Spin up new environments on
demand
 Stand-up/tear-down as part of
test automation
 Much better usage of tin –
easier to share across projects
and teams
24

CONNECT WITH US:
Beyond That: Into the Future
 Opportunity: What else can
we do with this automation?
 Think big for the future!
 Increasingly hands-free
automation
 Self scaling environments
 Detect shortage
 Provision new environment
 Lay on top app and config
 Self healing environments
 Detect issue
 Provision different version
 Apply known fix
25

CONNECT WITH US:
Our Approach
11/14/2016 26

CONNECT WITH US:
Practice Agility!
 The key to getting started and keeping going.
27
Some might… We did…
Start by envisioning a big
program of work.
Started simple – Then build on initial
success, now into 4th year of continued
improvement – best multi-year projects are
based on a small success.
Static delivery plan
Good principles of agility – Do something,
course correct, adapt to changing needs,
focus on strategic plan but still keep agility to
respond to immediate need
Fix-it-all in first 6 months
Pick an area to improve - Don’t “boil the
ocean”. We started by just fixing Continuous
Integration.

CONNECT WITH US:
Focus on Practices and Benefits!
 Focus on what is important.
28
Just focus on putting
in place new tools and
technologies.
Consider the practices – We thought
through what it is we’re trying to do and
chose the right tools and technologies
to support.
Get caught up in
“what is hot in
DevOps”.
Focus on benefit – As we intended on
keeping resource/budget small, we had
to make sure that everything we did
gave “best bang for buck”.
Practices
& Benefits

CONNECT WITH US:
Cross Boundaries!
 Important to keep eyes open to wider opportunities.
29
Stick with the areas
they are
comfortable with
e.g. Java.
Cover it all! - Thrived on figuring out how
to automate assembly and deploy of the
trickier areas e.g. database, repository-
based dev, service registries. Ensure that
all parts of the app are included.
Just focus on
developers and
operations teams.
Consider all involved teams – Any role
that is involved in releasing apps,
including DBAs, testers.

CONNECT WITH US:
Keep Efficient!
 DevOps sounds like it needs a big team.
30
Go it alone, or
outsource it all.
Best of both worlds – Prolifics involvement allows
BMI to focus on writing and testing apps, while
BMI release engineering team has still had direct
control of app releases.
Create a big team to
solve a big problem.
Small as possible - Achieved almost unbelievable
levels of efficiency with team of just 2.5 FTEs
(and one of those is a manager). Low cost,
disproportionate benefits.
Either do no training,
or overcommit to
training.
Pragmatic value-driven training - Train by
example, support with JIT workshops and Wiki-
based documentation.

CONNECT WITH US:
Leverage Expertise!
 Working with a partner has allowed BMI to make much
more progress.
31
“Not invented here!”
– lets build our own
automation.
Reuse! - Leveraged automation from the
Prolifics DevOps Labs. Huge difference in
how much has been achieved as well as
level of quality.
Give us 6 months
and we’ll have our
own automation
development
capability.
Engage specialists - Prolifics DevOps
labs efficiencies have allowed us to be
responsive to BMIs needs – we’ve
created the engine.

CONNECT WITH US:
What Does Success Look Like?

CONNECT WITH US:
Planning and Prioritizing Work
 Apply agility to planning
 Start from backlog
 Release plans track what we’re
currently working on
 Thrice weekly touchpoints
 Focus on prioritizing:
 Hot current issues
 Hot opportunities for
improvement
 Strategic improvements
 Constantly maximizing value!
33

CONNECT WITH US:
Prototyping New Automation
 Uncover risk, uncover opportunity
 Seek to understand before we
automate
 What are artifacts? Where do
artifacts live? What are APIs/tools?
 Approach
 Do it manually. Then automate
manually – command-line. Then
script automation
 Examples!
 Involve experts and teams - Demo
and get feedback
34
DeveloperWorks Knowledge Center
Samples Try it all out!

CONNECT WITH US:
Implement Automation in RTC
 Great automation development support
 Configuration management of automation
and example apps
 Inject automation (with changes) into
example build workspace
 Build engine to do test runs – attach logs,
perform activity logging
 Link up development work to plan items in
release plan
 Record of progress, issues, solutions
 JBE simple automation engine – use Ant
as standard
35

CONNECT WITH US:
Architect for Ease of Maintenance and
Extensibility
 Well factored plugin model
 Ant-based core
 Standard assembly and deploy steps atomic
steps
 Additional atomic steps e.g. stop app,
undeploy app, set default app, etc.
 Composite steps are mini-processes
 Reusable utilities in common
 Package up libs where appropriate
 Helper scripts where appropriate
 Interface with APIs and tools appropriate to
target platform
 Version to handle upgrades of target
platforms
36
APIs and Tools
Target Platform
Core Automation Plugin
build.xml
libs
scripts
Atomic
•assembleApp
•deployApp
•etc.
Composite
•assembleAndDeployApp
common.xml

CONNECT WITH US:
Wrap Plugins for UrbanCode Deploy
 Port deploy automations to
enterprise deployment platform
 Simple set of wrapper files around
proven automation
 Benefits
 Keep orchestration (deploy
processes) flexible
 One button deploys for multi-
component, multi-tier apps
 Snapshot whole app and move
through environments
 Gates, approvals, etc.
37
UCD Plugin
Core Automation Plugin
build.xml
libs
scripts
Atomic
•assembleApp
•deployApp
•etc.
Composite
•assembleAndDeployApp
common.xml
plugin.xml
wrapper scripts.groovy
Atomic
• deployApp
• etc.
component template.json • component
processes

CONNECT WITH US:
Continuous Integration and Provisioning
of Plugins
 Practice what we preach – optimized
delivery of plugins
 Automate plugin assemble and deploy
 Core plugins and UCD plugins
 Reduce time-to-test
 Run in a continuous integration cycle to
detect issues early
 Combine with running of automation steps
to test working
 Feedback loop
 Snapshot plugins – promote to PROD
38
DEV/TEST PROD
DEV/TEST
Deploy
PROD
Deploy
DEV/TEST
Deploy
PROD
Deploy
JBE
UCD
Server
J
B
E
J
B
E
J
B
E
UCD
Server

CONNECT WITH US:
Containerize build agents
 Creating a flexible DevOps
infrastructure with Docker
 Build agent host installs take time
 Docker gives simple way of
 Codifying install process
 Shipping around “golden” images
 Starting/stopping as many instances
(containers) as you wish
 Keeping build machine stacks
isolated
 Provisioning new machines
“instantly”
 Prevent build machines from
clogging up with installs and
temporary files
39
Before:
After:

CONNECT WITH US:
How Has This Helped?

CONNECT WITH US:
What has been achieved?
 Automated assemble and
deploy across 10
technology platforms.
 Accelerated delivery
pipelines for 11+ application
development streams.
 Oversight of deployments
for all involved projects.
 Core Build/Release team of
2.5 FTEs
41

CONNECT WITH US:
What Benefits are Being Seen?
 Able to reinvest upwards of 10
resources into more valuable
activities.
 Delivery is sooner.
 Testers are able to test more.
 Quality of code delivered is higher.
 Visibility of delivery has increased.
 Outages due to deployment error
close to eliminated.
42

CONNECT WITH US:
Impact on the Business, Customers, and
Us?
 Able to keep up with exploding
number of music delivery channels.
 Able to deliver more features that
make it easier to do business.
 Giving a better and more reliable
quality of service to customers.
 Collected royalties result in a
healthier music industry, meaning
better music for all of us!
43

CONNECT WITH US:
Thank You
Your Feedback is Important!
Access the InterConnect 2016 Conference Attendee
Portal to complete your session surveys from your
smartphone,
laptop or conference kiosk.

IBM InterConnect 2016 Greg Hodgkinson 2238 Thriving DevOps at BMI (Prolifics)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to IBM InterConnect 2016 Greg Hodgkinson 2238 Thriving DevOps at BMI (Prolifics)

Similar to IBM InterConnect 2016 Greg Hodgkinson 2238 Thriving DevOps at BMI (Prolifics) (20)

Recently uploaded

Recently uploaded (20)

IBM InterConnect 2016 Greg Hodgkinson 2238 Thriving DevOps at BMI (Prolifics)