Description: For the past many years, the focus of the industry was on improving quality by writing better code and using automation and engineering. These efforts did show results but we are still far from an illusive zero-defect state while fulfilling expected time to market. In order to further boost software quality improvement efforts, the focus has now shifted to improving testing processes. Test Maturity Model integration (TMMi) is a detailed model for test process assessment and improvement. TMMi staged model contains stages or levels that determine the maturity of an organization and lays down roadmap for moving from one maturity level to next. As the processes evolve, an organization passes from adhoc (L1) to Optimized levels (L5). Application of TMMi will lead organizations to highly mature test process, reduced rework (about 10%), increase in defect removal efficiency (about 85%), higher productivity (about 10 test cases per day or 15% higher) and faster time to market. Learning Objectives: 1. How can your organization improve the effectiveness of testing 2. How can you sustain testing maturity 3. What is the improvement roadmap for the implementation journey to improve testing processes 4. Why TMMI is preferred to CMMI for any testing organization

1. Suresh Chandra Ganesh Bose
Anagha Mahajan
SureshChandra.GaneshBose@cognizant.com
Anagha.Mahajan@cognizant.com
HOW TO TAKE ORGANIZATIONS
TO HIGHER TESTING MATURITY

2. Biography
2
Biography
Suresh Chandra Bose, Ganesh Bose is a Manager Consulting at Cognizant Business Consulting practice. Suresh
is based out of Austin, Texas and has been in the IT Industry for more than 16 years with vast Consulting
experience in various industries and executed Strategic initiatives for various Fortune 100 companies in the
areas of PMO, PPM, Process Consulting, Program Management, TMMI assessment/implementation,
Organization Strategy, Test Consulting and CIO/Governance Dashboard/Metrics for various clients across the
globe.
SureshChandra.GaneshBose@cognizant.com
Anagha Mahajan is a Senior Consultant with Cognizant’s Business Consulting practice. Anagha has diverse
global experience with multiple Fortune 500 companies in Technology, Telecommunications and Healthcare.
Her specialties include Due Diligence, Process Improvement, IT Transformation, IT Strategy, IT Governance and
Organization Maturity Improvement. Anagha has an MBA from Babson college and is a Six Sigma Black Belt
Certified professional.
Anagha.Mahajan@cognizant.com

3. Overview of TMMi
Need for TMMi Assessment
TMMi Assessment Maturity Rating Methodology
Contents
3
Benefits of TMMi Process Transformation
Case study
Testing Maturity: Need of the hour
Organizational Implementation Approach
Critical Success Factors for Implementation

4. Everybody is not Michelangelo
4
Become Michelangelo first, for everyone else must follow process.

5. 5
Testing Maturity: Need of the hour
Improves
Time to
Market
Cost of
Supplier
Contracts
Cycle Time
Cost of
Delivery
Ability to plan
Testing
Standardizes
Testing Method
Implementation of
Industry best
practice
Product
Quality
Defect
tracking
metrics
Ease of
integrating Test
into the delivery
model
Process
improvements
Reduces
Unpredictability
Rework

6. 6
 TMMi is developed by the TMMi Foundation
 TMMi (Test Model Maturity Integration) is a detailed model for test process improvement and is positioned
as being complementary to CMMi
 Assessment is performed taking into consideration the level the Organization want to be appraised of
 Contains guidelines and framework for test process improvement
 Has five maturity levels for process evaluation in systems and software engineering
 Addresses cornerstones of structured testing as lifecycles, techniques, infrastructure and organization
 By far the most popular model in the Industry for benchmarking Testing practices
TMMi Maturity Levels
Overview of TMMi
1-
Initial
2-
Managed
3-
Defined
4-Measured
5-
Optimization

7. 7
Need for TMMi Assessment
Standardized and
Optimized testing
process along with
higher degree of
process compliance
Effective monitoring and
control mechanism to
mitigate schedule and
cost overruns and
quality issues
Showing value to our
customers as IT getting
more complex
Assess efficacy of
support functions viz.
Environment, Build
Management etc. ,for
testing activities
Attesting the current
testing process against
Industry standards to
enhance credibility in
market place
Test Performance goals
with Metrics and
Measures to predict
performance and drive
STLC improvements
Integration of industry
best practices with the
testing process to
achieve business
objectives

8. 8
• Capability Maturity Model Integration
• Developed by Carnegie Mellon University
• Focused on end to end SDLC practices
• Limited focus on Test Improvements
• Has both staged and continuous
representation
• Has 3 others frameworks – CMMi for
Development, CMMi for Acquisition and CMMi
for Services
• Test Maturity Model Integration
• Developed by TMMi Foundation
• Focused only on Testing processes/ practices
• Limited focus on Non-Testing Improvements
• Has only staged representation
• No additional TMMi frameworks
CMMI
SVC
CMMI
DEV
CMMI
ACQ
Comm
on
Core
PAs
Comparison - CMMi and TMMi

9. 9
5. Optimized
•Test Process Optimization
•Quality Control
4. Measured
•Test Measurement
•Product Quality Evaluation
•Advanced Peer Reviews
3. Defined
•Test Organization
•Test Training Program
•Test Life Cycle and Integration
•Non-Functional Testing
•Peer Reviews
2. Managed
•Test Policy and Strategy
•Test Planning
•Test Monitoring and Control
•Test Design and Execution
•Test Environment
1. Initial
•Testing is a chaotic,
•Undefined process
Continuous Process
Improvement
Process Standardization
Stable Process
Can Repeat
Tasks
Ad Hoc
Informal
Unpredictable
Quantitatively Managed
P
r
o
c
e
s
s
M
a
t
u
r
I
t
y
Improvement Roadmap for Implementation

10. Assessment Plan
Our Assessment Approach
10
Cognizant will employ its accredited method comprising of documentary evidence reviews and interviews to ensure adequate
coverage of the generic and specific goals and practices across various process areas of TMMi, as illustrated below for Level 2 and 3
assessment. Timelines change based on the maturity assessment levels
Engagement Planning RecommendationsTMMi Assessment
Informal Assessment
Survey (Optional)
Documents Review
Week A Week B Week C
Identify Strengths &
Improvement
Opportunities
Findings Report
Recommendations &
Implementation Plan
- Deliverable
Interviews
Initial discussions to
understand requirements
Present assessment
approach and understand
objectives & constraints
Agreement on Proposal
and onboard resources
Develop Proposal for
Engagement Kick-off
Determine Maturity
levels
Develop Recommendations
& Implementation Plan
Implementation Order
& Priority
Best practice sharing
across projects/
applications
Management
Thoughts

11. TMMi Assessment Maturity Rating Methodology
11
Specific &
Generic Goals
RatingMechanism
Test Policy & Strategy Test Planning
Test Process
Optimization
Establish Test Policy Establish Test Strategy
Institutionalize
Defined Process………….
Process
areas
Specific&Generic
Practices
ProcessAreas
Specific&Generic
Practices
Questionnaire
/Client Survey
Responses
Interviews
Documents
Review
Overall Process
Area Maturity
Level
Enablers
Inhibitors
Instance
Characterization
Organizational Unit
Characterization
Goal Level Rating
Maturity Levels Calibration
………….
Specific&Generic
Goals
* Ratings represented above are Illustrative Only

12. TMMi Maturity Level Calibration Guidelines
Process Areas are designated the appropriate maturity levels based on the following guidelines
 Fully Achieved -
 Convincing evidence of process compliance
 Systematic and widespread implementation of process
 No obvious weakness in in distribution, application and results of this process exists
 process achievement is between 85 and up to 100%
 Largely Achieved –
 Significant evidence of process compliance
 systematic and widespread implementation of process
 Minor weakness in distribution, application and results of this process exists
 process achievement is between 50 and up to 85%
 Partially Achieved –
 Some evidence of process found
 Process exhibit significant weaknesses, be incomplete, not widespread, or inconsistent in application or results.
 process achievement is between 15 and up to 50%
 Not Achieved -
 Little or no evidence of process
 Process achievement is between 0 and up to 15%
 Not Rated - Any supporting goal is Not Rated
 Not Applicable - The process area is considered not to be in the scope of the assessment or applicable to the organizational unit by the Lead
Assessor
12

13. Improving and Sustaining Testing Maturity
13
Laying the ‘Foundation’
Initiate ‘Transformation’
Drive the recommendations aligned
with desired QA outcomes, measure
effectiveness and stabilize
Prioritized ‘Optimization’*
Quality Policy Test Policy Test Approach
Governance
Communication
Responsibilities
Implementation of testing capability
recommendation
Implementation of tailored processes in
projects
Streamline key Functions
Pilot key testing recommendations in
identified projects
Refine test process improvements (if
required)
Standardize testing at organization level
*Review Checkpoint – evaluation of effectiveness of implemented recommendations to assess and take
corrective actions/refine if required
Focus on transforming the key focus
areas and kick start rolling out
processes and testing enablers
Implement the foundation layer
focussing on setting up test
Function and Lifecycle
Implementation
Foundation
TransformationOptimization

14. TMMi Process Transformation - Projected Savings
14
A high level snapshot of the improvements in specific process areas have been categorized into the following themes and levers, whose
benefits and saving potential have been highlighted in the subsequent slides
Level 1. Initial 2. Managed 3. Defined 4. Measured 5. Optimized
Savings & ROI
ROI over previous Maturity
Level
0.0% 26.1% 40.1% 13.8% 8.2%
Total Return on Investment % 0.0% 26.1% 55.7% 61.8% 64.9%
End to End Process
Framework
Quality Assurance
Strategy
• Well defined quality gates
• Quality assurance governance
• End to End validation life cycle processes
• Clear responsibility and accountability for test delivery
Validation Risk
Profiling
Process Maturity
• Implementation and institutionalization
• Compliance evaluation
• Risk and impact tracking
• Preventive measures
Continuous
Improvement
• Causal Analysis
• Incremental improvements

15. Benefits of TMMi Process Transformation
TMMi Levels Improvement Areas Cost Quality
Speed to
Market
2. Managed
Test Policy and Strategy
Test Planning
Test Monitoring and Control
Test Design and Execution
Test Environment
3. Defined
Test Organization
Test Training Program
Test Life Cycle and Integration
Non-Functional Testing
Peer Reviews
4. Measured
Test Measurement
Product Quality Evaluation
Advanced Peer Reviews
5. Optimized
Test Process Optimization
Quality Control

16. 16
Critical Success Factors for Implementation
Sponsor
• Continued commitment and focus from Project sponsor and Executive management
• Attention and support of all Stakeholders (including Business, Development, Testing)
Participation
• Active participation of practitioners throughout the engagement
Feedback
• On-going support and feedback from all members of the organization
Communication
• Appropriate planning and communication

17. 17
Case study: TMMi assessment and implementation for a major technology company
“Conduct assessment of current testing processes against TMMi level 3 maturity, build and implement
transformation roadmap that addresses the gaps between current and target end state - taking into
consideration tactical and strategic opportunities to achieve level 3 maturity level”
 Cognizant was invited to carry out a consulting engagement to:
o Benchmark current test processes against TMMi Level 3 maturity
o Identify the target test process
 Based on the findings, Cognizant defined a roadmap addressing the gaps and also recommended a phased
approach to improve the testing process with an objective of transforming the client to highly mature testing org
DELIVERABLES:
 Current State Report – Detailed overview of the Current strengths and best practices & Maturity scoring by focus
groups within each IT groups
 Target State Report with Implementation Roadmap - Overall solution summary & High-level recommendations
across focus areas & Implementation approach, roadmap along with timelines
 Testing Process Artifacts – Test Policy and Strategy, TCP, Test Design Techniques, Traceability, Role Skill matrix,
Test Environment, NFT, Risk Management and Defect Management.
 Cognizant proposes a phased approach to transform client’s validation group to TMMi Level 3 Maturity.
Wherein first key process improvement capabilities first defined and then implemented in an
Agile/continuous fashion
 Key Solution Themes identified for achieving improvements are:
o Organization – Setup audit function, NFT team and configuration team.
o Lifecycle and Processes – Refine test planning, employ test design techniques, maintain bi-directional
traceability and utilize TCP for estimation
o Tools, Infrastructure and Support Functions – Streamline risk management, formalize review process,
facilitate standardized training and test environment management.
BENEFITS SNAPSHOT
 Team Size: 5 Consultants (2 onsite, 3 offshore)
 Duration: 4 weeks (assessment) + 12 weeks
 Weekly status report shared with key stakeholders
that details activities completed in current week,
plan for next week, overall status and risks
 Technical Working Group (TWG): A group of cross-
functional individuals from client organization was
formed to act as change agents, provide feedback
and review and approve Cognizant artifacts
 Train the trainers model: Training conducted for
identified key resources on processes/capabilities
and they will in turn train their respective team
members
ASSESSMENT HIGHLIGHTS
ASSESSMENT APPROACH
KEY RECOMMENDATIONS
 Scalable organization with improved ability to roll
out predictable quality outcomes
 Standardized processes within the organization
 Reduced defect leakage and quicker time to
market
 End to end metrics platform for unified reporting
to stakeholders
 Optimized automation / regression approach to
increase overall application quality
 Reduced cost of testing ownership
 Improved levels of productivity

18. References
18
• TMMi Foundation Reference Model
http://www.tmmi.org/pdf/TMMi.Framework.pdf

19. Contact Us @
19
Suresh Chandra Ganesh Bose
SureshChandra.GaneshBose@cognizant.com
www.linkedin.com/in/gsubose/
https://twitter.com/gsubose
Anagha Mahajan
Anagha.Mahajan@cognizant.com
https://www.linkedin.com/in/anaghamahajan/
https://twitter.com/anaghamahajan

20. 20
Questions?

21. 21