O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Maturing your path toward DevOps with Continuous Testing

192 visualizações

Publicada em

nterest in Continuous Testing has been growing for 5 years now—yet the more we talk about it, the more polarized the discussion becomes. Complicating the conversation is the fact that Agile and DevOps are both driving the need for Continuous Testing, but both require distinctly different things from a quality perspective.

Join me for a lively discussion on what’s really required for Continuous Testing in the context of Agile and DevOps. Join Eran Kinsbruner, author of Continuous Testing for DevOps Professionals, as he explores:


How DevOps and Agile change the game for testing


Which elements of Continuous Testing are absolutely essential for Agile and DevOps


The top myths, misconceptions, and mistakes surrounding Continuous Testing 


Strategies for measuring Continuous Testing progress and ROI

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Maturing your path toward DevOps with Continuous Testing

  1. 1. • • • • • • Twitter: @ek121268 (https://twitter.com/ek121268) Blog: http://continuoustesting.blog LinkedIn: https://www.linkedin.com/in/eran-kinsbruner-4b47a81/
  2. 2. 5/24/2018 3© 2018, Perfecto Mobile Ltd. All Rights Reserved. Continuous Testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as rapidly as possible.
  3. 3. State of DevOps Report, 2018 - DORA
  4. 4. State of DevOps Report, 2018 - DORA
  5. 5. There Are Patterns for “Unstable” Test Automation 80% of issues have a pattern52% success rate 10% of devices, causing 80% of lab issues Lab 25% Orches tration 25% Scripts & FW 50% FAILURE REASON Objects Codding Time Other Scripts & FW issues Device in use No Device Orchestration issues Networking Stability Lock Other Lab issues What’s wrong With my Scripts What’s wrong With my Lab What’s wrong With my Executions
  6. 6. 5/24/2018 7© 2018, Perfecto Mobile Ltd. All Rights Reserved. 3 2 1 3 2 ● ● ● ● ● ● ● ● ●
  7. 7. 1. What’s the test engineer’s gut feeling 😊 2. Risk calculated as probability to occur and impact to customers 3. Value – does the test provide new information and, if failed, how much time to fix? 4. Cost efficiency to develop – how long does it take to develop and how easy is it to script? 5. History of test – volume of historical failures in related areas and frequency of breaks Source: Angie Jones
  8. 8. Insights into the CI Pipeline Risk/Focus Area Mapping Summary Report List Single Test Report Visual Validations Noise reduction through error/failure-classification
  9. 9. • Pairing / Coaching • Use the right object identification strategy • Use the right test framework to work with • Measure test efficiency within the CI • Risk-based approach to test automation • Continuous test data analysis and improvement
  10. 10. • How fast are testing activities moving, and what is slowing down these activities? • Test flakiness • Test duration • % of automated vs. manual tests • Application quality measurements • # of escaped defects and in which areas • MTTD – mean time to detection of defect • Build quality • Pipeline efficiency measurements • # of user stories implemented per iteration • Test automation as part of DoD across iterations • Broken builds with categories • CI length trending • Lab availability and utilization • Quality costs measurements • Operational costs, lab availability issues • Cost of hardware/software • Costs of defects by severity and stage
  11. 11. Key ML Use Cases In Test Automation • Recognize objects • Transcribe speech – Accessibility • Make quality related decisions based on data • Identify Trends and/or Patterns • Security use cases – Identify signatures e.g. 11/20/2018 14© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  12. 12. Mindset and Workflow Changes in Test Automation 11/20/2018 15© 2015, Perfecto Mobile Ltd. All Rights Reserved. # Category 1 Test Authoring 2 Test Maintenance 3 Test Execution 4 Test Analysis 5 Tools Maturity 6 Available Integrations 7 Req. Test/Dev Skillset 8 Testing Env. 9 Testing Types 10 Supported App Types Test Engineers/Developers Manual Testers (Developers?) Traditional Test Automation Define Manual Flows, BDD Style, etc. Test Code & Reusable Functions (Java, JS, etc.) Define Objects, POM, use Object Spy Structured Page Based Test Steps Define Visual Validations and Assertions Longer Time To Develop, Complex Changes required pro-actively, SCM Supported Configure Env. (TestNG Data Provider), Execute Locally/CI/Cloud Flexible, OSS, Commercial High, Including Guidelines, Doc’s Plenty, OSS, Defect Management, etc. Medium-High IDE API, Load, Functional All Types ML Based Test Automation Record Test Flows (No Coding – in most cases) Groups Reusable Functions Objects Generated On-The-Fly (transparent to user) Flow Based Scenarios Part of Test Authoring Recording/Authoring Shorter Time, Advanced Capabilities Self-Healing/Correction Automatically handled/Object Scoring/Local SCM Controlled Configure Env., Execution Management Built-In, Locally/CI/Cloud Relies on ML/AI Tool Vendor in Most Cases Emerging Technology, Web More Advanced Than Mobile Most capabilities built-in, some exists Low-Medium ML UI Mostly Functional (E2E) & API Mostly Web

×