© 2013 IBM Corporation
Shifting Left Approach and Practices
Paul Bahrs Chief Architect, Emerging Technologies, IBM Software, Rational
Nov 6, 2014

Who is this guy?
•Paul is the Chief Architect for emerging technologies, focused on new or acquired solutions and accelerating early client successes
•Today he works with IBM clients with their adoption of DevOps, Cloud, and Quality solutions to meet market and business demands
•Paul is the published on DevWorks, ibm.com, a presenter at industry/IBM conferences, patent author and supported open standards
Paul Bahrs

Market shifts are fundamentally changing what and how we do IT
3
Software Development
Customers
Development
Testing
Deployment
Macro Business Environment Volatile economic and changing regulatory environments require businesses to adapt quickly to changing market conditions
Empowered Users
The exponential increase in empowered users drives expectations for higher quality customer experiences and the need for continuously updated software
Technology Trends
Mobile, cloud, big data, social, agile and delivery analytics increase application complexity while promising better accessibility, flexibility and agility
Business Owners

Develop /
Test
Steer Deploy Operate
IBM DevOps – most comprehensive capabilities
Addresses bottlenecks and waste across the delivery lifecycle
Continuously plan, measure and
bring business strategy and
customer feedback into the
development lifecycle.
Enable collaboration between
business, development, and QA
to deliver innovative, quality
software continuously.
Reduce the cost of testing while helping
development teams balance quality and
speed.
Deliver software to customers
and internal users faster and
more frequently with better
quality, lower cost, and reduced
risk.
Understand and accommodate the
user perspective to achieve service
levels with better visibility and
continuous feedback across the
entire software lifecycle.
Continuous
Business Planning
Collaborative
Development
Continuous
Testing
Continuous Release
and Deployment
Continuous
Monitoring
Continuous
Customer Feedback
& Optimization
Provide the visual evidence and full
context for analyzing customer behavior
and pinpointing pain points.

© 2013 IBM Corporation
Some Observations

6-12 Month Delivery Cycles Are Still the Norm

Where do Testers want to spend their time?
Creating Automated Tests
Performing Exploratory Tests
Executing Automated Tests
Designing Tests
Planning Tests
Reviewing Test Results
Running Pre-defined Tests
Maintaining Automated Tests
I want to spend MORE time…
I want to spend LESS time…
Creating Reports
Maintaining Automated Test Tools
Configuring Test Tools
Creating Test Data
Re-running tests
Investigating, Documenting and Submitting Defects
Setting Up Test Labs

What are deployment engineers doing?
*Data based on UrbanCode customer survey

Bottlenecks affecting Test
Delays for Business Scope
Provision, Deploy, Test,
Delivery, Integration, Build
Production Releases
Continuous Changes from Business

Water Fall Projects
Biz-Dev-Test-Ops are organizational aligned and interact formally
Integration test phase immediately follows Development to “enable testing”
Testing follows INT and is usually impacted by delays or bottlenecks
–Time delays to scope project
–Time to move changes through test to production
Change from business is continuous throughout the project
Feature deliveries weekly or bi-weekly
Integrations are resource intensive and manual
QA and beyond require formal builds
Build and Deploy process are governed but usually not automated nor continuous.
Managed or virtual environments are not necessarily impacted
Deployments are performed by developers in Dev/INT and more formally (by organizations) in other environments.
Feedback to features / stories may still be delayed by weeks or provided by the developer
The need for INT is relative to the testing performed during iterations. QA is still formal test within interface context

Water-Agile-Fall Projects
Biz-Dev-Test-Ops are organizational aligned and interact formally except for construction
Integration test phase may still follow Development to “enable testing”
Iterations achieve planning flexibility, but most testing still occurs after development
–Some departments may use periodic iterations to integrate, build, deploy, test
–Developers are usually burdened with doing their own testing during iterations
Change from business is continuous throughout the project
Feature / story deliveries during iterations
Integrations may still be resource intensive but performed more often
Iteration testing requires periodic builds to verify but may not test
Build and Deploy process are either distinct or heavily interdependent
Managed environments require 2-3 weeks to 2-3 months to prepare
Deployments are performed by multiple organizations with different processes/technologies
Feedback to features may be delayed by weeks or months or not at all due to time
INT and QA are first time to perform test within interface context

Agile Projects
Biz-Dev-Test are usually collaborative for Dev purposes. Ops may still be organizational aligned and interact formally
Integration and QA test phase needs are reduced if testing performed during construction
Bottlenecks begin at
–Business planning to support iterative or accelerate release cycles
–Ability of environments, application deployments to keep up with agile teams
–Test team to support iteration testing – early and expand testing to exploratory/performance/loading
Change from business is continuous throughout the project
Stories delivered during iterations
Integrations are ongoing between team members, components
Builds services may be provided for individual, team and applications
Build and Deploy process are run often to verify and to run test/scanning during process
Environment availability is dependent on complexity of changes and capacity of ops
Time through the “post construction” pipeline is determined by quality of code, risk and velocity of pipeline.
Feedback to developers to dependent on periodicity, type and quality of testing
Testing periodicity is dependent on the provision, deploy process capacity

On to Shifting Left

What is Shift Left Testing?
"Shift Left" is a DevOps practice that provides an effective means to perform testing with or in parallel to development activities.
–Development, test and operations work together to plan, manage and execute automated and continuous testing to accelerate feedback to developers
–The rate of the feedback is determined by an organization’s desired velocity.
–Technology is used to automate processes and virtualize components.
Testing may be performed as a part of the development process or as a service running in parallel to development activities. In either case, shifting left accelerates feedback to developers and improves the quality of code delivered to QA.
Shift left is an essential capability for Agile teams but may be successfully leveraged by all development teams.

What Shifts Left?
Perspectives
–Testing: Begin test execution earlier
–Development: Act on feedback to achieve value
–PM: Resource allocation to address feedback
–Infrastructure: Shift left environments available early, capacity drives availability
–Deployment: Accelerated capacity to meet test periodicity
Activities:
–plan, create and automate integration test cases in advance
–support a periodic cycle of integration testing for the components/applications under change
–prioritize, process, and resolve feedback by development teams within the feedback period
–plan, automate and deploy components, applications and virtual services
–plan, create and virtualize component services relevant to the development activities
–execute delivery, integration, build, deploy, testing, feedback and correction during the defined periodicity.
Whatever testing you can perform shifts left - Integration testing could be your first step as a means to provide a means to prepare for a formal QA and reduce bottlenecks associated with Integration testing post development.

Why Some Clients Shift Test Left
Move Defect Curve to left
–Reduce costs because defects are cheaper earlier (Dev/Test teams cannot test adequately)
–Change enough to find more defects early (Dev/Test teams have a long wait time to get the right environment)
–No change to project management processes (We have too many configurations to be tested. There is no consistent way to spin the right environments in time without errors)
Reduce Issues in Production
–Dramatic quality improvements earlier in pipeline to reduce risk to production
–Continue quality maturing through to production
–Support across technology
–Promotion criteria should ensure quality is met
Prepare for Continuous Delivery
–Reduce bottleneck in QA and set stage for continuous delivery
–Start QA on delivery of first feature
–Ensure automation supports usage for Developer, Team, Application, Project
–Ensure the proper SLA’s and Reuse are supported
Support for Agile teams
–Ability for Agile teams to test daily
–Ability to provide and act on feedback daily
–Ability to scale as Agile adoption scales

Where should you change?
Environment
The “What”
The “How”
The “Where”
The “Verb”
•Application Development
•Application Deployment
•Environment Provision
•All types of Testing

Steer
Develop / Test
Deploy
Operate
Scaled
Reliable
Repeatable
Practiced
Practice improvements
Define release with business objectives Measure to customer value
Optimize applications Use enterprise issue resolution procedures
Standardize and automate cross-enterprise Automate patterns-based provision and deploy
Manage data and virtual services for test Deliver and integrate and build continuously
Link objectives to releases Centralize Requirements Management Measure to project metrics
Link lifecycle information Deliver and build with test Automate testing Embed Quality Reporting
Document objectives locally Manage department resources
Manage Lifecycle artifacts Schedule SCM integrations and automated builds Test following construction
Plan and manage releases Standardize deployments
Monitor resources consistently Collaborate Dev/Ops informally
Plan and source strategically Dashboard portfolio measures
Automate problem isolation and issue resolution Optimize to customer KPIs continuously
Improve continuously with development intelligence Test Continuously Leverage Quality Tends
Manage environments through automation Provide self-service build, provision and deploy
Adopting IBM’s approach: https://ibm.biz/BdDaMe
Plan departmental releases and automate status Automated deployment with standard topologies
Monitor using business and end user context Centralize event notification and incident resolution

Considerations
Overall
–Identify feedback velocity and means to measure it
–Scope of automation pipeline (build, deploy, test)
–Decide which changes will improve team’s success (versus introduce functionality)
–Continuously improve and plan for next steps (they will be needed)
Software Configuration Management
–Delivery and integration processes velocity and measures
–Complexity that causes bottlenecks and delays
Build
–Time to build – number of application level builds
–Accessibility for developer, team, application
–Periodicity of builds and disposition of results
Deploy
–Time to deploy
–Rate of deployment
–Scalability to extend to post deployment actions
–Cross platform, approved versions and virtual services
Test
–Provide complete or virtual environments each event
–Only start testing early only then improve
–Test automation should follow immediately

Lifecycle Measurements
2008
2010
2012 – 2014
Total Improvement
Project Initiation
30 days
10 days
2 days
28 days
Groomed Backlog
90 days
45 days
On-going
89 days
Overall Time To Development
120 days
55 days
3 days
117 days
Composite Build Time
36 hours
12 hours
5 hours
700 %
BVT Availability
N / A
18 hours
< 1hour
17 hours
Iteration Test Time
5 days
2 days
14 hours
4 days
Total Deployment Time
2 days
8 hours
4 hours -> 20 minutes
2 days
Overall Time To Production
9 days
3 days
2 days
7 days
Time Between Releases
12 Months
12 Months
3 Months
9 Months
Innovation / Maintenance
58% / 42%
64% / 36%
78% / 22%
+20% / -20%
Double-digit revenue growth, increased client adoption, improved client satisfaction
How IBM Rational Cloud Hosted Products have improved!

A Waterfall Client’s Experiences with Shift Left
Focus was on shifting integration test left, but not boiling the ocean on change. The practices that would be affected included code changes, delivery and integration, build automation, deployment automation. Test practices would merely execute earlier
Outcomes
Rework reduction: First time in 3 years to impact defect rate going into QA – 90% reduction.
Practice impact: Demos of UI and retrospective improved requirements understanding and business acceptance. No pilot project ever reported as medium or high risk during pilot.
Work reduction: Over 50% of testing was executed first week in QA almost 100% success (normally done in weeks with almost 50% defects)
Collaboration: Qualitative improvement of end to end testing. Faster issue resolution and cross team coordination. Automated build, deploy, unit test/scan and virtual service activation processes.

First Steps – Integration Test Feedback
Environment
The “What”
The “How”
The “Where”
The “Verb”
•Application Dev
•Code coverage
•Static Scans
•Code Reviews
•Security Scans
•Two/Screen
•Code deliveries
•Change Integration
•Binary management
•Documentation
•Technology maintenance
•Process execution
•Extensibility
•Speed, repeatability
•Scalability
•Environment consistency
•Levels of Delays
•Resource requirements
•Error prone processes
•Documentation
•Agility
•Availability
•Manual processes
•Automated processes
•Unit testing
•Regression testing
•Integration testing
•Requirements coverage
•Code coverage
•Test Data
•Virtual services/stubs

Shift Left Planning Workshop
Goals and Assessment
•Client Capabilities: Current Status
•Business Goals for Improvement
•Solution Capability Oriented
•People, Practices, Technology, information
Capability Priorities
•IBM Best Practices for Solution Capabilities
•Discussions to refine Objective Capability
•Prioritize Incremental Capabilities
•Vision, User Value, Pain and Complexity
Executive Sponsor Review
•Review Outcomes
•Assumptions & Risks
•Gain concurrence
•Identify next steps
Solution Improvement Roadmap
•IBM Best Practices for Adopting Solution
•Identify Key Milestones
•Roadmap activity to define actionable plan
•Define Quick Win Pilot
Day 1
Day 2
Day 3
Day 4
Current State Assessment
Objective & Prioritized Capabilities
Adoption Roadmap
Draft Results
Detailed 1-3 month roadmap for initial win. High level roadmap for remainder of initiative. Executable week following the workshop
Theme
Activities
Objective

Yes, we have tools that help
Rational Team Concert
–Continuous Integration
UrbanCode Deploy
–Application Deployment Automation
Rational Test Virtualization Server
–Service Virtualization

Now what?
Check out our Practices Self Assessment Tool to assess your current practices in context to Shift Left
–Ibm.biz/devops-practices-assessment
Review our Shift Left Practices @ Jazz.net
–Jazz.net/shift_left_practices
Contact us to discuss our workshop to plan your Quality improvements
–ratlsvcs@us.ibm.com
Reach out to me for more discussion
–@paulbahrs
–pbahrs@us.ibm.com
25

Next step - Shift Monitoring Left
26

© 2013 IBM Corporation
www.ibm.com/devops

© 2013 IBM Corporation
© Copyright IBM Corporation 2014. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
www.ibm.com/devops