Mais conteúdo relacionado

Apresentações para você(20)

Similar a Shift Left - Approach and practices with IBM(20)


Mais de IBM UrbanCode Products(20)


Shift Left - Approach and practices with IBM

  1. © 2013 IBM Corporation Shifting Left Approach and Practices Paul Bahrs Chief Architect, Emerging Technologies, IBM Software, Rational Nov 6, 2014
  2. Who is this guy? •Paul is the Chief Architect for emerging technologies, focused on new or acquired solutions and accelerating early client successes •Today he works with IBM clients with their adoption of DevOps, Cloud, and Quality solutions to meet market and business demands •Paul is the published on DevWorks,, a presenter at industry/IBM conferences, patent author and supported open standards Paul Bahrs
  3. Market shifts are fundamentally changing what and how we do IT 3 Software Development Customers Development Testing Deployment Macro Business Environment Volatile economic and changing regulatory environments require businesses to adapt quickly to changing market conditions Empowered Users The exponential increase in empowered users drives expectations for higher quality customer experiences and the need for continuously updated software Technology Trends Mobile, cloud, big data, social, agile and delivery analytics increase application complexity while promising better accessibility, flexibility and agility Business Owners
  4. Develop / Test Steer Deploy Operate IBM DevOps – most comprehensive capabilities Addresses bottlenecks and waste across the delivery lifecycle Continuously plan, measure and bring business strategy and customer feedback into the development lifecycle. Enable collaboration between business, development, and QA to deliver innovative, quality software continuously. Reduce the cost of testing while helping development teams balance quality and speed. Deliver software to customers and internal users faster and more frequently with better quality, lower cost, and reduced risk. Understand and accommodate the user perspective to achieve service levels with better visibility and continuous feedback across the entire software lifecycle. Continuous Business Planning Collaborative Development Continuous Testing Continuous Release and Deployment Continuous Monitoring Continuous Customer Feedback & Optimization Provide the visual evidence and full context for analyzing customer behavior and pinpointing pain points.
  5. © 2013 IBM Corporation Some Observations
  6. 6-12 Month Delivery Cycles Are Still the Norm
  7. Where do Testers want to spend their time? Creating Automated Tests Performing Exploratory Tests Executing Automated Tests Designing Tests Planning Tests Reviewing Test Results Running Pre-defined Tests Maintaining Automated Tests I want to spend MORE time… I want to spend LESS time… Creating Reports Maintaining Automated Test Tools Configuring Test Tools Creating Test Data Re-running tests Investigating, Documenting and Submitting Defects Setting Up Test Labs
  8. What are deployment engineers doing? *Data based on UrbanCode customer survey
  9. Bottlenecks affecting Test Delays for Business Scope Provision, Deploy, Test, Delivery, Integration, Build Production Releases Continuous Changes from Business
  10. Water Fall Projects Biz-Dev-Test-Ops are organizational aligned and interact formally Integration test phase immediately follows Development to “enable testing” Testing follows INT and is usually impacted by delays or bottlenecks –Time delays to scope project –Time to move changes through test to production Change from business is continuous throughout the project Feature deliveries weekly or bi-weekly Integrations are resource intensive and manual QA and beyond require formal builds Build and Deploy process are governed but usually not automated nor continuous. Managed or virtual environments are not necessarily impacted Deployments are performed by developers in Dev/INT and more formally (by organizations) in other environments. Feedback to features / stories may still be delayed by weeks or provided by the developer The need for INT is relative to the testing performed during iterations. QA is still formal test within interface context
  11. Water-Agile-Fall Projects Biz-Dev-Test-Ops are organizational aligned and interact formally except for construction Integration test phase may still follow Development to “enable testing” Iterations achieve planning flexibility, but most testing still occurs after development –Some departments may use periodic iterations to integrate, build, deploy, test –Developers are usually burdened with doing their own testing during iterations Change from business is continuous throughout the project Feature / story deliveries during iterations Integrations may still be resource intensive but performed more often Iteration testing requires periodic builds to verify but may not test Build and Deploy process are either distinct or heavily interdependent Managed environments require 2-3 weeks to 2-3 months to prepare Deployments are performed by multiple organizations with different processes/technologies Feedback to features may be delayed by weeks or months or not at all due to time INT and QA are first time to perform test within interface context
  12. Agile Projects Biz-Dev-Test are usually collaborative for Dev purposes. Ops may still be organizational aligned and interact formally Integration and QA test phase needs are reduced if testing performed during construction Bottlenecks begin at –Business planning to support iterative or accelerate release cycles –Ability of environments, application deployments to keep up with agile teams –Test team to support iteration testing – early and expand testing to exploratory/performance/loading Change from business is continuous throughout the project Stories delivered during iterations Integrations are ongoing between team members, components Builds services may be provided for individual, team and applications Build and Deploy process are run often to verify and to run test/scanning during process Environment availability is dependent on complexity of changes and capacity of ops Time through the “post construction” pipeline is determined by quality of code, risk and velocity of pipeline. Feedback to developers to dependent on periodicity, type and quality of testing Testing periodicity is dependent on the provision, deploy process capacity
  13. On to Shifting Left
  14. What is Shift Left Testing? "Shift Left" is a DevOps practice that provides an effective means to perform testing with or in parallel to development activities. –Development, test and operations work together to plan, manage and execute automated and continuous testing to accelerate feedback to developers –The rate of the feedback is determined by an organization’s desired velocity. –Technology is used to automate processes and virtualize components. Testing may be performed as a part of the development process or as a service running in parallel to development activities. In either case, shifting left accelerates feedback to developers and improves the quality of code delivered to QA. Shift left is an essential capability for Agile teams but may be successfully leveraged by all development teams.
  15. What Shifts Left? Perspectives –Testing: Begin test execution earlier –Development: Act on feedback to achieve value –PM: Resource allocation to address feedback –Infrastructure: Shift left environments available early, capacity drives availability –Deployment: Accelerated capacity to meet test periodicity Activities: –plan, create and automate integration test cases in advance –support a periodic cycle of integration testing for the components/applications under change –prioritize, process, and resolve feedback by development teams within the feedback period –plan, automate and deploy components, applications and virtual services –plan, create and virtualize component services relevant to the development activities –execute delivery, integration, build, deploy, testing, feedback and correction during the defined periodicity. Whatever testing you can perform shifts left - Integration testing could be your first step as a means to provide a means to prepare for a formal QA and reduce bottlenecks associated with Integration testing post development.
  16. Why Some Clients Shift Test Left Move Defect Curve to left –Reduce costs because defects are cheaper earlier (Dev/Test teams cannot test adequately) –Change enough to find more defects early (Dev/Test teams have a long wait time to get the right environment) –No change to project management processes (We have too many configurations to be tested. There is no consistent way to spin the right environments in time without errors) Reduce Issues in Production –Dramatic quality improvements earlier in pipeline to reduce risk to production –Continue quality maturing through to production –Support across technology –Promotion criteria should ensure quality is met Prepare for Continuous Delivery –Reduce bottleneck in QA and set stage for continuous delivery –Start QA on delivery of first feature –Ensure automation supports usage for Developer, Team, Application, Project –Ensure the proper SLA’s and Reuse are supported Support for Agile teams –Ability for Agile teams to test daily –Ability to provide and act on feedback daily –Ability to scale as Agile adoption scales
  17. Where should you change? Environment The “What” The “How” The “Where” The “Verb” •Application Development •Application Deployment •Environment Provision •All types of Testing
  18. Steer Develop / Test Deploy Operate Scaled Reliable Repeatable Practiced Practice improvements Define release with business objectives Measure to customer value Optimize applications Use enterprise issue resolution procedures Standardize and automate cross-enterprise Automate patterns-based provision and deploy Manage data and virtual services for test Deliver and integrate and build continuously Link objectives to releases Centralize Requirements Management Measure to project metrics Link lifecycle information Deliver and build with test Automate testing Embed Quality Reporting Document objectives locally Manage department resources Manage Lifecycle artifacts Schedule SCM integrations and automated builds Test following construction Plan and manage releases Standardize deployments Monitor resources consistently Collaborate Dev/Ops informally Plan and source strategically Dashboard portfolio measures Automate problem isolation and issue resolution Optimize to customer KPIs continuously Improve continuously with development intelligence Test Continuously Leverage Quality Tends Manage environments through automation Provide self-service build, provision and deploy Adopting IBM’s approach: Plan departmental releases and automate status Automated deployment with standard topologies Monitor using business and end user context Centralize event notification and incident resolution
  19. Considerations Overall –Identify feedback velocity and means to measure it –Scope of automation pipeline (build, deploy, test) –Decide which changes will improve team’s success (versus introduce functionality) –Continuously improve and plan for next steps (they will be needed) Software Configuration Management –Delivery and integration processes velocity and measures –Complexity that causes bottlenecks and delays Build –Time to build – number of application level builds –Accessibility for developer, team, application –Periodicity of builds and disposition of results Deploy –Time to deploy –Rate of deployment –Scalability to extend to post deployment actions –Cross platform, approved versions and virtual services Test –Provide complete or virtual environments each event –Only start testing early only then improve –Test automation should follow immediately
  20. Lifecycle Measurements 2008 2010 2012 – 2014 Total Improvement Project Initiation 30 days 10 days 2 days 28 days Groomed Backlog 90 days 45 days On-going 89 days Overall Time To Development 120 days 55 days 3 days 117 days Composite Build Time 36 hours 12 hours 5 hours 700 % BVT Availability N / A 18 hours < 1hour 17 hours Iteration Test Time 5 days 2 days 14 hours 4 days Total Deployment Time 2 days 8 hours 4 hours -> 20 minutes 2 days Overall Time To Production 9 days 3 days 2 days 7 days Time Between Releases 12 Months 12 Months 3 Months 9 Months Innovation / Maintenance 58% / 42% 64% / 36% 78% / 22% +20% / -20% Double-digit revenue growth, increased client adoption, improved client satisfaction How IBM Rational Cloud Hosted Products have improved!
  21. A Waterfall Client’s Experiences with Shift Left Focus was on shifting integration test left, but not boiling the ocean on change. The practices that would be affected included code changes, delivery and integration, build automation, deployment automation. Test practices would merely execute earlier Outcomes Rework reduction: First time in 3 years to impact defect rate going into QA – 90% reduction. Practice impact: Demos of UI and retrospective improved requirements understanding and business acceptance. No pilot project ever reported as medium or high risk during pilot. Work reduction: Over 50% of testing was executed first week in QA almost 100% success (normally done in weeks with almost 50% defects) Collaboration: Qualitative improvement of end to end testing. Faster issue resolution and cross team coordination. Automated build, deploy, unit test/scan and virtual service activation processes.
  22. First Steps – Integration Test Feedback Environment The “What” The “How” The “Where” The “Verb” •Application Dev •Code coverage •Static Scans •Code Reviews •Security Scans •Two/Screen •Code deliveries •Change Integration •Binary management •Documentation •Technology maintenance •Process execution •Extensibility •Speed, repeatability •Scalability •Environment consistency •Levels of Delays •Resource requirements •Error prone processes •Documentation •Agility •Availability •Manual processes •Automated processes •Unit testing •Regression testing •Integration testing •Requirements coverage •Code coverage •Test Data •Virtual services/stubs
  23. Shift Left Planning Workshop Goals and Assessment •Client Capabilities: Current Status •Business Goals for Improvement •Solution Capability Oriented •People, Practices, Technology, information Capability Priorities •IBM Best Practices for Solution Capabilities •Discussions to refine Objective Capability •Prioritize Incremental Capabilities •Vision, User Value, Pain and Complexity Executive Sponsor Review •Review Outcomes •Assumptions & Risks •Gain concurrence •Identify next steps Solution Improvement Roadmap •IBM Best Practices for Adopting Solution •Identify Key Milestones •Roadmap activity to define actionable plan •Define Quick Win Pilot Day 1 Day 2 Day 3 Day 4 Current State Assessment Objective & Prioritized Capabilities Adoption Roadmap Draft Results Detailed 1-3 month roadmap for initial win. High level roadmap for remainder of initiative. Executable week following the workshop Theme Activities Objective
  24. Yes, we have tools that help Rational Team Concert –Continuous Integration UrbanCode Deploy –Application Deployment Automation Rational Test Virtualization Server –Service Virtualization
  25. Now what? Check out our Practices Self Assessment Tool to assess your current practices in context to Shift Left – Review our Shift Left Practices @ – Contact us to discuss our workshop to plan your Quality improvements – Reach out to me for more discussion –@paulbahrs – 25
  26. Next step - Shift Monitoring Left 26
  27. © 2013 IBM Corporation
  28. © 2013 IBM Corporation © Copyright IBM Corporation 2014. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.