#ATAGTR2018 Presentation "Test Transformation - A Journey" By Seema Kohli and Sangeetha Kashyap

#ATAGTR2018
Test Transformation
Seema Kohli(IBM) &
Sangeetha Kashyap(IBM)
27th September 2018

#ATAGTR2018
As a author of this presentation I/we own the copyright and confirm the originality of the content. I/we allow Agile testing alliance to use the content for social media
marketing, publishing it on ATA Blog or ATA social medial channels(Provided due credit is given to me/us)
Agenda
 Overview and Objectives
 Test Transformation Methodology
 Transformation Focus Areas
 Benefits Accomplished and Road ahead

#ATAGTR2018
Overview and Objectives

#ATAGTR2018
Governance & Support
Stakeholder Management Innovation Council Dashboard Reporting
API &
Performance
Testing
Hadoop & ETL
Testing
Process
Team
Web & MF
Testing
Automation
Testing
Project
Management
Service Catalogue
QualityManagementCoordination
RequirementsEnvironmentsTestDataTSS
ServiceManagement
DefectMgmtEstimatingWorkIntakeDemandMgmt
Innovation & Value
Knowledge Management Continuous ImprovementMethodologyAssets & Accelerators
Metrics & Reporting
Automation TM & Architect
People
Functional Test Managers
Data Testers
MF & Web Test Specialist
API & Perf Test Specialists
Automation Test Specialists
Estimation Tool (Excel )
Tools/Process
ALM, Defect Management
API Parasoft
Performance LoadRunner
UFT, Query Surge
Automation Feasibility Analysis
Projects and Portfolio
Large Banking Account - associated with IBM for 9 years and with TCoE for last 3 years
Organization View

#ATAGTR2018
Test Transformation - Objectives
 Enable continuous reduction of the overall cost of quality and establish
effective control over testing budgets
 Reduce duration of current test planning, preparation and execution cycles /
phases and establish effective control over testing timelines
 Improve the overall quality of applications and systems via increasing
effectiveness and efficiency of testing services provided by TCoE

#ATAGTR2018
Test Transformation Methodology

#ATAGTR2018
Test Transformation - Methodology
Assessment TMMi Assessment Gap Analysis &
Recommendations
Presentation
to the Client
Detailed
Planning Engagement Exploration
Solution
Outline
Detailed
Transformation
Plan
Present Plan
to Client
Continuous
Improvement Rollout
People
Enablement BAU
Pilot & Deploy
Solution
Feasibility with
PoC People Enablement
Pilot Solution
Implementation Deploy
Monitor &
Measure
Roadmap
Identification of
Transformation
areas
Prioritization
based on
business value
Client Sign-off
on RoadmapTeam Setup
Monitor &
Measure
Revision &
Refinement
Finalization

#ATAGTR2018
Assessment
KEY RECOMMENDATIONS• Perform a
maturity &
effectiveness
assessment of
TCOE)
• Develop
baselines of
test cases,
automation
and defects
• Identify gaps in
maturity
• Bridge
Assessment
outcomes to
TMMI maturity
model
• Provide
Recommendati
ons for
improvement
• Develop a
Transformation
road-map
FINDINGS
• TCOE is consistent and
utilizes structured
processes and templates
• Focused on defect
detection but not really
enabling prevention
• Performs SIT but performs
only limited business
scenario validation
• Project based rather than
application based
• There are a lot of testing
reports generated, but
there is a lack of quality
metrics with the
associated governance
• Levels of automation
execution are low and
automation cannot be
created quickly enough to
make a difference
• Regression testing is
inadequate and should be
strengthened
IBM Rating = 2.1
. 2.6
2.0
1.8
2.3
1.9
Process &
Methodology
Culture &
Organization
Skills
Tools &
Technology
Measurement &
Controls
2.0
Process &
Methodology
Culture &
Organization
Skills
Tools &
Technology
Measurement &
Controls
TMMI Rating = 3
• Implement a QC of Excellence
model with a balance on both
detection and prevention.
• Develop an automation strategy to
increase the efficiency and
effectiveness of automation within
the TCOE
• Implement governance processes
between the TCOE and PDLC
stakeholders to illustrate results /
value
• Implement a metrics program
which provides insight into key
quality and testing metrics to drive
process change and shift left
• Perform POCs for additional tools
to drive increased productivity and
efficiency
• Adopt Test Case optimization to
increase the quality of test cases

#ATAGTR2018
Roadmap
 A set of tools / methods / solutions identified for transformation
 Prioritization of identified areas based on business value and immediacy of impact
 Transformation Team setup ; establish stakeholder interfaces

#ATAGTR2018
Key Challenges
 Alignment with different stakeholders to get buy-in on the solution/methodology
 Resistance to Change
 Divergent views and interests
 Availability of right-skilled resources and SMEs at an optimal cost and in required
timeframe
 Understanding of current system and business processes
 Skilled in new technology areas identified for transformation
 Ability to explore different solutions and approaches
 Infrastructure support and availability
 Access for installing and working on tools in client network
 Ports and firewall issue resolution during solution deployments in client
network
 License procurement and availability

#ATAGTR2018
Critical Success Factors
Critical Success Factors Comments
Project Prioritization Right prioritization of solutions based on client and business
needs
People Alignment Buy-in on proposed changes and new solutions from
delivery, process and client teams
Client Partnership Client commitment - a key enabler
People Enablement People were enabled in the new areas of technology and
methods with planned trainings and on the job learning
Budget Committed budget for transformation requirements
Skills Practitioners with required skills for solution development/
enhancements
Measurements Ability to quantify benefits to stakeholders
Several factors have contributed to the success of our transformation
journey. Key among these are listed below.

#ATAGTR2018
Detailed Planning
Engage Explore
Outline Plan

#ATAGTR2018
Pilot and Deployment
PoC for
Technical
Feasibility
Pilot on
Business
Application
Measure
Impact
and deploy
Enable
People

#ATAGTR2018
Continuous Improvement
Transition to
Delivery
Establish
Support
Model
Monitor
Measure
Report
Refine
People
Enablement
and SMEs
Support Model to maintain
and enhance the solution on
ongoing basis
Ensure right skills are
onboarded in the team
and SMEs are identified

#ATAGTR2018
Test Transformation Focus Areas

#ATAGTR2018
Test Transformation Technical Components
• Excel based template converted to centralized web based tool. Controlled data entry
and automated calculations. Role based estimation processWeb-based Estimation Tool
• Developed using Tableaus. Instant access to monthly dashboard; - portfolio / project
views for financial, project pipeline / status, resource, service quality dataTCOE Metrics - CIO Dashboard
• Instant access to current and historic AFA results in Confluence; Objective feasibility
analysis; Streamlined process
Automation Feasibility Analysis –
Confluence-Solution
•Streamlined capacity planning and resource availability/ work allocation viewsCapacity Planner: Process and MPP-based
Tool
• Focus tool (CTD) – accelerated test design for test optimization and increased test
coverage, OTFA – First pass automation thus increasing automation penetration
IGNITE - CTD and script less automation for
Web applications (OTFA)
•First of its kind automation framework built using the Parasoft tool. RAML file is an
input to CTD, the output of which is input to the automation frameworkAPI Test Automation Framework
•Continuous improvement of test automation effectiveness and efficiency
Test Automation Council
•Redefined and re-aligned R & R, empowered the offshore leads to perform work such
as AFA and Portfolio Assessment, re-skilled team to align with transformation strategy
New Operational Model for Test
Automation

#ATAGTR2018
Transformation Focus Areas

#ATAGTR2018
Process Automation

#ATAGTR2018
Application Automation
IBM IGNITE
 Consolidation of functional and automation tester roles
 Test case optimization and improved test coverage
 First pass or progressive automation, thus improving automation penetration
 Easy to use, requiring minimal automation knowledge
API Test Automation Framework
 First of a kind solution for automating API testing with Parasoft and extendible to
work with other tools like SOAPUI
 Integrates with IGNITE CTD to provide optimized test cases

#ATAGTR2018
Methodologies and Process

#ATAGTR2018
Benefits and Road ahead

#ATAGTR2018
Qualitative Benefits
Qualitative Benefits for Client Qualitative Benefits for IBM
Client TCOE supported by IBM is uniquely
positioned and recognized as the most
efficient TCOE amongst all other TCOEs in
the client organization
Customer satisfaction index went up in 2018
Q1 from the previous rating.
Only TCOE that has sophisticated reporting
and metrics published. This has uniquely
positioned client AVP as one of the top
testing leaders in the client organization.
Client is referenceable. Client participated in
Gartner survey of 2017 because of the
transformation activities. They have agreed
to participate in 2018 Gartner survey as
well.
Increased efficiency, speed to market and
quality of deliverables
Process methods and people have led to
qualitative benefits and cultural shifts in the
account that now thrives on continuous
improvement

#ATAGTR2018
Quantitative Benefits
Solution / Tool Quantitative Benefits
Test Estimation Tool  Solution helped achieve effort reduction of 33.5
hours (Manual effort of 38 hours to 4.5 hours with
the solution)
CIO Dashboard  Solution helped achieve effort reduction of 48
hours (Manual effort of 80 hours to 32 hours with
the solution)
AFA – Automation Feasibility
Analysis – Confluence based
Solution
 Solution helped achieve effort reduction of 3 hours
(Manual effort of 4 hours to 1 hour with the
solution)
IGNITE  Effort savings of up to 30% in the overall testing
lifecycle by using IGNITE
 Test Optimizations between 40 to 70%
API Test Automation Framework  30% effort savings compared to the manual testing
for an API application
 Optimized test coverage – gain of 28% test data
combinations in pilot leading to more
comprehensive testing

#ATAGTR2018
Journey Continues
 Client has derived benefits as committed in the program - IBM has
exceeded the expected results
 This is a highly repeatable customer journey for many other clients looking
for Testing Transformation. The challenges and the solution provided by
IBM will be of great interest to the testing community
 The journey now continues in the year 2018 with enhancements to
introduced solutions and introduction of new transformation opportunities

#ATAGTR2018 Presentation "Test Transformation - A Journey" By Seema Kohli and Sangeetha Kashyap

Recomendados

Recomendados

Mais conteúdo relacionado

Mais de Agile Testing Alliance

Mais de Agile Testing Alliance (20)

Último

Último (20)

#ATAGTR2018 Presentation "Test Transformation - A Journey" By Seema Kohli and Sangeetha Kashyap