http://www.hcltech.com/enterprise-transformation-services/overview~ More on ETS The purpose of this paper is to illustrate the significance of incorporating CoQ principles in software solutions and to evaluate its impact on business. It also highlights HCL’s capabilities in analyzing IT projects and implementing the CoQ approach across business domains, which bear ample testimony to the fact that CoQ is a business opportunity and not a cost centre. This paper will be of interest to IT professionals who are exploring ways to help their companies speed up time‐to‐market and cut operational cost.

1. Cost of Quality
Needs Your Attention Not Sometimes
But Always
Authors
Manas Chakraborty, Symu Koul, & Suresh
Sundararajan, ETS-IV&VS
HCL Technologies, Chennai
Dr Usha Thakur, ATS Technical Research
HCL Technologies, Chennai

2. CoQ: Needs Your Attention Not Sometimes But Always
© 2010, HCL Technologies Ltd.
June 2010
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 2

3. Contents
Introduction ............................................................................................................................................................ 4
Purpose ................................................................................................................................................................... 4
CoQ and Its Relevance ............................................................................................................................................ 5
CoQ Service from HCL ............................................................................................................................................. 7
CoQ as an Integral Part of SDLC ........................................................................................................................ 12
Benefits ................................................................................................................................................................. 14
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 3

4. Introduction
In and of themselves tools and technologies are not very useful and when they end up in the hands of people
who do not understand their significance, they are as good as not having any. By the same token a software
solution is not very valuable in itself, but it can serve as a powerful tool for enabling and transforming business
operations; in fact, it has the potential of giving impressive business results.
Of late, IT roles, activities, solutions, and services have come under the scanner of business executives and are
being assessed for the extent to which they create (business) value i.e., help company executives in achieving
strategic objectives such as giving their company a competitive edge, increasing profit margins, cutting overall
operational costs, retaining existing customers and obtaining new ones, increasing customer satisfaction,
improving internal communication, and speeding up time-to-market for software products and services. It is,
therefore, not surprising that the impact of software solutions and services is increasingly being measured in
terms of business criteria such as Return on Investment (ROI) and Return on Quality (ROQ). Moreover,
businesses are no longer willing to work within the confines of IT silos; they want IT to adapt easily to the
dynamic needs of business without compromising on quality.
If the goal of software solutions is to improve the way a business is run so that it becomes more profitable and
healthy, then Cost of Quality (CoQ) is a key approach for achieving that kind of improvement. The principles of
CoQ are well established in the manufacturing industry, given its long history.1 Their relevance in determining
the quality of software solutions has been understood only in the last couple of decades, mostly in the context
of the costs incurred as a result of delivering poor quality software. The main idea behind a CoQ approach is to
increase revenue and decrease operational cost by reducing rework and maximizing opportunities.
Purpose
The purpose of this paper is to illustrate the significance of incorporating CoQ principles in software solutions
and to evaluate its impact on business. It also highlights HCL’s capabilities in analyzing IT projects and
implementing the CoQ approach across business domains, which bear ample testimony to the fact that CoQ is
a business opportunity and not a cost centre. This paper will be of interest to IT professionals who are
exploring ways to help their companies speed up time-to-market and cut operational cost.
1
For an interesting read on this see, Matthew Littlefield, "The Cost of Quality: Benchmarking Enterprise Quality
Management," (2007) http://www.hertzler.com/php/portfolio/white.papers.php [April 2010]->represents when the site
was accessed.
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 4

5. CoQ and Its Relevance
Any discussion on the subject of CoQ leads one to acknowledge the contributions of Juran, Gryna and Crosby2
who emphasized that it is more economical to pay attention to quality early in the lifecycle of a product (by
incurring prevention and appraisal costs) than to wait for products to fail (for non conformance) and then fix
them (thereby incurring failure costs). Realizing its significance, the Software Engineering Institute (SEI) has
made quality an integral part of its Capability Maturity Model Integration (CMMI) best practices.3 Furthermore,
many organizations have made an effort to improve the quality of their software by integrating the principles
of Six Sigma.4
According to Crosby,
 Quality means conformance to requirements, not goodness
 Quality is achieved by prevention, not appraisal
 Quality has a performance standard of Zero Defects, not acceptable quality levels
 Quality is measured by the Price of Non-conformance™, not indexes5
In other words, CoQ refers to the cost that companies undertake to rectify the mistake they made and have to
correct it. By the same token CoQ refers to the cost companies incur to ensure that the software solutions they
deliver to their clients are free of defects. To achieve that goal, companies need to incur costs (read ‘invest’) in
the form of resources, tools and activities throughout the lifecycle of software to identify and prevent defects
as well as potential failures. The seriousness of problems arising from software errors was very well highlighted
2
For their earliest contributions the CoQ debate see J. Juran and F. Gryna, Quality Control Handbook 4th ed., McGraw-
Hill, New York, 1988. See also P. Crosby, Quality Is Free: The Art of Making Quality Certain McGraw-Hill, New York, 1979.
3
For an implementation of quality principles, see Dennis R. Goldenson, et al., "Why Make the Switch: Evidence about the
Benefits of CMMI," (2004) http://www.sei.cmu.edu/library/assets/evidence.pdf [April 2010].
4
While adherence to Six Sigma principles in software has its share of critics, they have been used for improving the
quality of software and reducing the overall CoQ. See, for instance, Dr. Bill Eventoff, "Applying Six Sigma to Software
Development: A Practical Guide," (2002) http://www.swqual.com/SQGNE/presentations/2004-
05/Eventoff%20May%202005.ppt [April 2010]. See also Maneesh Aggarwal, "Six Sigma Meets Software Development,"
http://www.isixsigma.com/index.php?option=com_k2&view=item&id=1307:&Itemid=49 [April 2010].
5
Cited in Kevin Weiss, "Biography - Philip B. Crosby: June 18, 1926 – August 18, 2001,"
http://www.philipcrosby.com/25years/crosby.html [April 2010]. For an interesting article on calculating the ROI of CoQ,
see Sandra A Slaughter, et al., "Evaluating the Cost of Software Quality," COMMUNICATIONS OF THE ACM August
1998/Vol. 41, No. 8. This paper is also available at http://www.cse.buffalo.edu/~hungngo/SCE-Papers/p67-slaughter.pdf
[April 2010].
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 5

6. by a study commissioned by the Department of Commerce's National Institute of Standards and Technology
(NIST) in the US in 2002.6 The study acknowledged that
[s]oftware is error-ridden in part because of its growing complexity. The size of software
products is no longer measured in thousands of lines of code, but in millions. Software
developers already spend approximately 80 percent of development costs on identifying and
correcting defects, and yet few products of any type other than software are shipped with such
high levels of errors. Other factors contributing to quality problems include marketing
strategies, limited liability by software vendors, and decreasing returns on testing and
debugging, according to the study. At the core of these issues is difficulty in defining and
measuring software quality. The increasing complexity of software, along with a decreasing
average product life expectancy, has increased the economic costs of errors. The catastrophic
impacts of some failures are well-known. For example, a software failure interrupted the New
York Mercantile Exchange and telephone service to several East Coast cities in February 1998.
But high-profile incidents are only the tip of a pervasive pattern that software developers and
users agree is causing substantial economic losses.7
In 2008 IDC conducted a survey among quality managers of 139 North American companies. The survey
respondents estimated that if 100% of software defects were addressed and remediated prior to production,
they would experience a 32% cost savings.8 In 2008 Original Software conducted a qualitative opinion survey of
senior IT executives and CIOs from organizations with average annual revenue of $5.8billion and an average IT
budget of $764 million. The survey revealed that over 60% of them felt a strong need to improve the quality of
software applications.9
6
See Michael Newman, "Software Errors Cost U.S. Economy $59.5 Billion Annually: NIST Assesses Technical Needs of
Industry to Improve Software-Testing," (2002) http://www.nist.gov/public_affairs/releases/n02-10.htm [April 2010]. See
also Bill Hoffman, "Battling Software Defects One Developer at a Time," (February 23, 2010)
http://www.itbusinessedge.com/cm/community/features/guestopinions/blog/battling-software-defects-one-developer-
at-a-time/?cs=39611 [April 2010].
7
Ibid. Similar concerns were highlighted by Herb Krasner. His research led him to conclude that "[t]ypical manufacturing
CoQ, rang[es] from 5 to 25 percent of company sales, [which] contrasts significantly with CoSQ. With the present state of
software engineering practice we can expect CoSQ to range from 10 to 70 percent of development costs. Even accounting
for the margin between production costs and sales, CoSQ appears to be roughly twice manufacturing CoQ." See Herb
Krasner, "Using the Cost of Quality Approach for Software," (February 2006), p. 7,
http://lifelong.engr.utexas.edu/pdf/sqi/cosq-article-new06.pdf [April 2010].
8
See Melinda-Carol Ballou, "Improving Software Quality to Drive Business Agility," (June 2008)
http://www.coverity.com/library/pdf/IDC_Improving_Software_Quality_June_2008.pdf [April 2010]. This survey was
sponsored by Coverity Inc.
9
See "Software Quality and Testing: A CIO Perspective," http://www.origsoft.com/whitepapers/ [April 2010].
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 6

7. CoQ Service from HCL
HCL’s CoQ service is about total quality management of software in the pre or post development phase. It is
most helpful in determining the structure of cost across the lifecycle of software and in developing and
implementing best practices strategies. This service includes:
 Thorough assessment of Quality Assurance (QA) activities and cost structure in four areas: planning,
engineering, review and testing, and rework
 Detailed gap analysis between corporate vision and existing QA practices in relation to where the
organization concerned wants to be
 Root-cause analysis of both costs: good and poor quality
 Recommendations for improving CoQ
 Identification of CoQ business cases
 Implementation of high quality standards
As illustrated in Figure 1, a good part of the CoQ exercise is contingent upon parameters used for determining
the quality of ‘good’ and ‘poor’ quality.
Figure 1: Cost of Quality Parameters
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 7

8. Cost of Good Quality: Appraisal activities, among others, measure, evaluate and audit products or services to
assure conformance to quality standards and performance requirements; their aim is to ensure that software
is free from defects and conforms to customer requirements. Prevention activities ensure that necessary
training, processes, tools, and hardware as well as resources are available for performing appraisal activities.
Since both the activities are pre-emptive measures, they are regarded as the cost of good quality.
Cost of Poor Quality: When detects (arising, for instance, from non-conformance to customer requirements)
are not caught during the development phase of software and are noticed just prior to delivery, they require a
great deal of re-work and re-testing effort. If software defects come to the surface after delivery, the cost of
review and root-cause analysis effort are significantly higher. Irrespective of when the failures come to light
(before or after delivery), they both reflect non-conformance to customer requirements, and are hence
referred to as cost of delivering poor quality software.
The value of being conscious about the cost of good and poor quality is fully realised only when, as shown in
Figure 2, the assessment data obtained from various stakeholders (at a client organization) is entered in HCL’s
CoQ analysis tool.
Figure 2: CoQ Modeling Results
Furthermore, as illustrated in Figure 3 and Figure 4, the HCL analysis tool allows for the quantification and
consolidation of quality parameters. Consequently, senior management at client organizations can then focus
on areas that require further improvement and develop a strategy for keeping the CoQ at acceptable levels.
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 8

9. Figure 3: Quality Parameters Reflecting As-Is Situation
Figure 4: Quality Parameters Reflecting a Reduction in CoQ
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 9

10. The HCL team uses an indexing system to make the results of the CoQ exercise meaningful to client
organizations. For instance, in the sample shown in Figure 5, the index number of 4, (which represents low
quality and high cost) was arrived at following a cost and quality assessment of testing practices at a client
location. The results allowed senior management to identify areas that required attention in order to reduce
cost and improve software quality.
Figure 5: CoQ Indexing
Based on its root cause findings, the HCL CoQ team identifies initiatives for closing the gaps between As-Is and
the desired objectives. As shown in the sample in Figure 6, the proposed initiatives are clubbed into solution
themes (group of initiatives), prioritized for ease of implementation, and then mapped to an IT strategy. The
HCL team also calculates the business benefits of implementing its proposed solution and suggests numerous
options for its implementation such as building the proposed solution in-house organically, partnering with a
third party to obtain the required skills, developing skills with third party via a co-sourcing/out-sourcing
strategy, or out-sourcing the entire solution to another company.
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 10

11. Figure 6: Theme Based Solution Proposition
As illustrated in the sample in Figure 7, the HCL team prioritizes the themes in its proposed solution and maps
them in a 2/2 matrix to show specific business benefits and ease of implementation. This exercise helps client
organizations to focus on recommendations that are not only easy to implement right away but also offer
business benefits (in terms of quality, cost, and productivity). Moreover, as and when required, client
organizations can extend the same recommendations to other projects in phases.
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 11

12. Figure 7: Business Benefits of Proposed Solution
CoQ as an Integral Part of SDLC
Although it is widely acknowledged that CoQ principles and practices should be used throughout the lifecycle
of software, most software teams find themselves walking a tightrope between the needs of business teams
(for faster time-to-market) and development teams (who require more time for comprehensive testing).
As shown in Table 1, in cases where development teams have the mandate to integrate CoQ principles and
practices throughout the lifecycle of software, HCL’s CoQ team can assist client organization in several
concrete ways to overcome challenges that are faced by software, testing teams, and allied service teams.
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 12

13. Testing Phase-Wise Activities Business Benefits HCL Deliverables Challenges
Phases to Client
Planning Requirements analysis and Reduction in effort Reports Ensuring the
implementation feasibility and cost to fix defects  Requirements analysis involvement of the
(discovered later)  Tool evaluation CoQ team right from
Identification of tools for test Documents requirements phase
management, defect management, Savings realized on  Testing estimation
test automation and performance tool cost and  Test strategy
testing rationalization of  Microsoft project plan
tools usage across  Non functional
multiple projects requirements
 Test scenario
identification
 Technologies and skills
set identification
High Level Design analysis and feasibility study Savings in effort and Reports Understanding
Test Design cost related to defect  Detailed design software functionality
Identification of: fix at early stage feasibility analysis
 impact to upstream and  High level test Ensuring the
downstream application and scenario and impact involvement:
aligning testing activities to analysis  Domain experts
cover all areas Documents  Performance
 test environment and test  Requirements engineering
data set-up Traceability Matrix experts
 Hardware and
System capacity planning and load software
modeling requirements and Test
Data control sheet for
gathering test data
Detailed Detailed test case preparation and Avoid spend on Upload test cases in test Having resources with
Test Design review of test cases (by business rework management tools skills set in testing
users/SMEs) to ensure complete techniques
coverage of test cases against
functionality
Performance scenario identification
Test Smoke testing to ensure basic Launch of new Reports Having resources who
Execution functionalities are working such features /application  Test summary are adept at
that the entire QA team can be within budget and on  Daily status addressing challenges
involved for testing all the features time  Defect status related to test
environment and test
System integration testing, data set-up
regression testing, user acceptance
testing, performance testing,
Identification of performance
bottlenecks and recommendations
fixes
Script execution in coordination
with other resources involved for
monitoring
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 13

14. Testing Phase-Wise Activities Business Benefits HCL Deliverables Challenges
Phases to Client
Support Defect verification and impact Increase in customer Documents Collecting data on
analysis satisfaction resulting  ROI based automation  customer
from better  Production defect satisfaction data
Update test cases related to defect performance of trend analysis  revenue
fix software generation
 penalties paid
Support for enhancement and Savings in effort and and support
impact analysis cost due and increase charges
in revenue due to
Calculation of Return on readily available test Having resources with
Investment ( ROI) based on automation advanced knowledge
application readiness, number of framework and ease of test automation
release and nature of of maintenance
enhancements
Customization of existing HCL test
automation framework requiring
minimal maintenance
Table 1: CoQ and SDLC
Benefits
If software products suffer from a high number of defects, low quality, and require a great deal of manual
effort and rework, then it is next to impossible to achieve business objectives such as speed up time-to-market
and lower operational cost. What the CoQ approach from HCL allows client organizations to do is not only
identify problem areas (which result in poor quality of software) but also implement a holistic plan of action
for achieving business objectives and monitoring its progress in a measurable manner on a continuous basis.
Cost of Quality: Needs Your Attention Not Sometimes But Always Page 14