Mais conteúdo relacionado

Apresentações para você(20)


Mais de QA or the Highway(20)


Agile Testing Transformation is as Easy as 1, 2, 3 by Michael Buening

  1. Agile Testing Transformation is as Easy as 1, 2, 3 Presented by: Mike Buening Venue: QA or the Highway 2019 Date: February 19, 2019
  2. 2 Currently serving as Director, Digital Assurance and Testing in the East Ohio Region for Sogeti US. Most recently working as a QA and Testing consultant with gas and electric utility companies in Columbus, OH. My focus is on testing discipline uplift in support of an agile transformation. Previous roles include information analyst, business analyst, test lead, project manager, test manager, and department head, over a 20 year IT career focused mostly on quality assurance of technology projects and products. I obtained a Ph.D. in biochemistry from The University of Notre Dame, which has nothing to do with IT but naturally aligns with quality assurance and testing. Science is all about acquiring information through experimentation. Software testing is exactly the same thing. During my first 30 years I spent time in Japan, Alabama, Ohio, Okinawa, Indiana, Kansas, South Dakota, and Indiana (again). Along the way I met my wife and brought her with me. For the last 20 years I have lived in Central Ohio, currently in Lewis Center with my wife, three kids, and two dogs. Outside work, my interests include music and living vicariously through my kids. Introduction
  3. 3 Agenda • What is Agile Testing Transformation? • A Roadmap for Transformation • Three Phases of Transformation • Workforce Transformation • Common Gotcha’s
  4. 4 What is Agile Testing Transformation? Manual Tests Through UI Automation Suites Unit Testing Automated UI Tests Automated Functional/API Tests Automated Unit Tests Manual & Exploratory Testing Agile Transformation Traditional (find bugs) Agile (prevent bugs)
  5. 5 Roadmap to Agile Testing Transformation • Establish TEC • Test Expertise • Test Process Standards • Metrics • Test Data & Environment Requirements Optimize Test Fundamentals • Functional and Non-functional test case automation • CI/CD build automation • Environment Virtualization • Test Data Management Implement Basic Automation • Analytics & AI • Model-Based Testing • Automated Test Design • AI-Driven Test Set Design Smart QA Automation and especially smart test automation is poised to bring about significant changes in the way QA and testing is done over the next two to three years and organizations need to have a strategy and roadmap in place if they want to reap its benefits. We recommend a phased approach in three stages, i.e. first, the optimization of testing, second, implementation of basic automation, and third, the adoption of intelligent and self-adaptive test automation solutions to make automation "smarter." 1 2 3
  6. 6 Step 1: Optimize Test Fundamentals • Establish TEC • Test Expertise • Test Process Standards • Metrics • Test Data & Environment Requirements
  7. 7 Test Fundamentals • Establish Test Excellence Center (TEC) • Maintains standards for processes and procedures • Supports tools used for testing • Provides specialty testing services • Provides consulting on best practices and troubleshooting complex testing concerns • Does NOT provide or manage testing staff – testing expertise is decentralized and assigned to individual teams • Supports developing solutions around business domains where data and environment requirements have commonality TEC Team 1 Team 2 Team 3 Team 4 Team 5 Team n Business Domain A Business Domain B IT
  8. 8 Unit Test Plan Unit Test System Test Acceptance Test The V Model Development Levels Test Levels Business Requirements Acceptance Test Plan System Requirements System & E2E Test Plan Integration Test Plan Design Documents Program Specifications Static Testing Dynamic Testing Integration Test Development IT Test Org Business Org Coding Solution Requirements Functional Reqs Non-Functional Reqs Regression Testing Performance Testing Operational Readiness Testing Application Operations Test Fundamentals
  9. 9 Test Strategy & Planning Test Design Test Execution Test Closure  Starts at project inception – waterfall, agile, hybrid, other  Enable strong business and functional requirements elicitation  Initiate work on test strategy and planning  Key roles: Test Lead  Refine requirements and enable strong solution designs  Use a variety of test design techniques using risk as a driver for coverage and depth of testing  Design and build automated testing framework and test scripts  Key roles: Test Analyst, Automated Testing Engineer  Manage test execution by a variety of resources  Monitor test execution status – adjust plans accordingly  Manage defect lifecycle including triage, resolution, retest cycles  Key roles: Tester, Test Lead  Summarize test results – quality of the software being tested; success and efficiency of the testing process  Build and groom reusable regression test suites  Key roles: Test Lead, Test Analyst Test Fundamentals
  10. 10 Test Strategy & Planning Test Design Test Execution Test Closure Test Strategy & Planning Time Time Waterfall Agile Test Design Test Execution Test Closure Strategy & Planning consider full project scope and anticipate all testing needs for complete project Strategy is a living, reusable, driver relevant for a standing product team Planning focuses on current and near term sprint objectives Design is requirements driven using risk to determine depth of coverage Design flows from story card acceptance criteria and definition of done. Automate everywhere it makes sense to do so. Testing follows the plan. Often reliant on manual execution, only automating where ROI is clear. Testing is tightly coupled with development and proceeds until definition of done is achieved and product owner is satisfied. Closure summarizes test results and provides reusable test assets to the application team. Closure proceeds via sprint review & retrospective. Automation is critical as this sprint’s functional tests are next sprint’s regression. 10
  11. 11 Step 2: Implement Basic Automation • Functional and Non-functional test case automation • CI/CD build automation • Environment Virtualization • Test Data Management
  12. 12 Implement Basic Test Automation  Automation of web services testing for internal and 3rd party calls  Virtualization of services for early integration testing and defect identifications  Cross browser testing with single script  Mobile testing capabilities across various Hardware and Software  Non Functional Testing tools and approaches  Orchestration platform for Performance Testing and Engineering  Centralized test management, reporting and monitoring  Leverage automation to minimize manual test case execution  Incrementally build and execute regression and ETE tests  End of sprint checkpoints for regression suite optimization and review  In-Sprint functional automation design and execution  Enable CI/CD pipeline Functional, Regression & DevOps Services Testing, Virtualization, Cross Browser and Mobile Testing Performance & Non Functional Testing
  13. 13 Step 3: Smart QA • Analytics & AI • Model-Based Testing • Automated Test Design • AI-Driven Test Set Design
  14. 14 Smart QA: Intelligent Analytics – “What Matters Most ?”  Tester efficiency  Test case analytics  Test requirements analytics  Test design analytics Test Management data  Root cause analysis  Defects by module  Defects by release  Defects found by tester  Defects introduced by developer Defect data  Heat Map analysis  Automated Incident Analysis  Smart Root cause analysis  Performance analytics Production Data How do I apply Predict what to test Predict efficiency of workforce Predict bottlenecks to release Predict speed of release Predict quality of release  Conduct Static/Dynamic code analysis  Generate Technical Debt, Code Coverage & Lines of Code Metrics Code data
  15. 15 Smart QA: Emerging trends envision an AI driven future for QA Role of BOTS in Testing • Integrate tests as part of DevOps pipeline • Test environment setup • Test data generation • Test automation Trends for AI in Testing • Intelligent automation • Predictive analytics • Descriptive analytics • Self-learning, cognitive platforms
  16. 16 Workforce Transformation
  17. 17 The tester role is split into two: SDETs and Quality Engineers. Quality Engineers with their deep product knowledge and test/quality domain from an E2E view can provide expertise & focus on what should be tested. SDETs become the go-to engineers for projects that need TECHNICAL expertise in test automation and integration. SDET constantly designs better and more creative ways to identify weak spots. SDET Responsibilities  Build frameworks and packages required to implement automation  Write code to test: UI and API level testing  Create code and tools so developers could test their own code  Design focused on testability, robustness, and performance – code review, component testing, continuous testing SDET Technology Skills  Selenium, JUnit, Nunit, TestNG, and other similar tools  Knowledge beyond Automation tools like QuickTest Pro, TestDirector, SilkTest  Webservice testing & virtualization  Agile/Scrum and DevOps knowhow  Basic knowhow with high-level OO language like Java | Python  RDBMS, SQL Quality Engineers Skills  Good depth of knowledge for the application  Domain, Manual Testing, UI Testing  Mobile/Cross browser testing  Automation Enabled testers  Basic BDD, Basic Java  Basic Continuous Integration Test Software Development Engineer in Test (SDET) vs. Quality Engineers
  18. 18 QA COE Independent Teams QADevelopment BA Build & Deploy QA TEC Hybrid Teams Initial State Silo Teams with Defined Roles & Responsibilities Intermediate State Hybrid teams, SDET as catalyst to drive transformation End State Integrated full stack teams Single Skill I Profile Teams A Primary and Secondary Skill T Profile Teams • User Story Analysis • Technical Design • Code Development • Code Review • Unit Testing • Build Support • User Story Analysis • Test Plan and Design – Manual, Automation, Data and Non Functional • Test Execution • Defect Management • Business Engagement • User Story • Acceptance Criteria Definition • Code Integration/Merge • Create Build • Deploy Build in Target • Build Checkout System Team Scrum 1 Scrum 2 Scrum N DEV/BA Overlap Zone SDET E2E Testing Performance DevOps TDMAccessibility TestingApp Scan QA TEC Integrated Team Pi Profile Team System Team Performance DevOps TDMAccessibility TestingApp Scan Scrum 2 Scrum N E2E Testing Cross Functional Team (Dev / BA / SDET) Scrum 1 • Business Engagement • Model Based Impact Analysis and Test Design • Test Harness Dev. • Code Development • Continuous Integration; Automated Build, Deploy • Sprint Demo Workforce Transformation: Evolution of Roles to Support this Journey
  19. 19 Common Gotcha’s • An emphasis on test automation without fundamental testing discipline in place leads to waste in the form of inefficiency. • Every functional test case does not need to be automated • Regression testing is still the largest and costliest testing type over the lifespan of a product • Functional test automation should minimize the cost and increase the speed of regression testing • Across the industry, test data and test environment management are the top blockers for effective progress in QA and testing. • Automated testing needs dependable testing environments • Automated testing needs consistent, well defined, test data • External dependencies can wreck havoc on Sprint commitments • The more the team can control their own destiny the more efficient they will be • Service Virtualization is a powerful tool used to decouple external dependencies • Support! All of this automated testing is software that needs run support. Don’t forget to build it into the plan and make sure the support staff are included in the workforce transformation.
  20. 20 Common Gotcha’s Automated UI Tests Automated Functional/AP I Tests Automated Unit Tests Manual & Exploratory Testing Fully Integrated End-to-End Business Process Testing Integration Testing - Functional, Performance, Regression for Impacted Systems The full scope of testing required for high quality delivery often needs additional considerations beyond what a team is capable of addressing, reasons include: • Knowledge • Skill • Environment/Data/Infrastructure • Access • Tools • Capacity • Time Coordination and input from people and systems outside the Scrum team might be needed to perform integration testing and end-to-end business process testing.
  21. 21 Common Gotcha’s When several Agile teams are sharing dependencies on common enterprise systems, testing becomes a cross-team challenge requiring a high degree of coordination and communication. Enterprise Data Sources This may seem at odds with Scrum’s emphasis on the team owning all the work needed to deliver to production. However, if each of the teams are testing the middle tier and enterprise data systems independently they will likely collide with each other, generating uncertainty which can delay delivery and introduce defects to production. Automated UI Tests Automated Acceptance Tests Automated Unit Tests Automated UITests Automated Acceptance Tests AutomatedUnitTests Automated UITests Automated Acceptance Tests AutomatedUnitTests Automated UITests Automated Acceptance Tests AutomatedUnitTests Enterprise Data Sources Middle Tier Integration
  22. 22 Summary • Organizations embarking on an Agile Transformation need to plan the testing transformation • Agile Testing Transformation should be built in a solid, risk-based, fundamental testing process • Establish basic functional and non-functional test automation coupled with a continuous integration process • Smart QA uses analytics, AI, and advanced automation techniques to optimize testing • Agile Testing Transformation requires organizational changes and workforce transformation