One of the really serious problems with modern software development is the fact that 99% of software development organizations equate Software Quality Assurance with Testing. This is a very costly misunderstanding. Quality Assurance is supposed to help reduce costs and help produce better software. Testing can only add to costs and it can only be done once something is built. So when testing starts, all the errors are already made and all the bad things that SQA is supposed to prevent have already occurred. Testing is pretty much the exact opposite of Quality Assurance.

1. Quality Through Measurement
Quality Assurance is Not Testing
Extracted From
Shrinking the Software Scrap Heap
Tom Walton
International Quality Conference 2005
Toronto, Canada,
October 5 to 7, 2005
tom_walton@videotron.ca
Shrinking the Software Scrap Heap reserved
All rights

2. SEI CMM Summary
Level
5
Optimizing
4
Managed
3
Defined
2
Repeatable
1
Initial
Characteristics
• Improvement fed back into process
• Data gathering is automated and used to identify weakest
process elements
• Numerical evidence used to justify application of technology
to critical tasks
• Rigorous defect -cause analysis and defect prevention
(Quantitative)
• Measured Process
• Minimum set of quality and productivity measures
established
• Process database established with resources to analyze its
data and maintain it
(Qualitative)
• Process defined and institutionalized
• Software Engineering Process Group Established to lead
process improvement
Key Challenges
• Still human- intensive process
• Maintain Organization at
optimizing level
Result
Productivity
&
Quality
• Changing technology
• Problem analysis
• Problem prevention
• Process measurement
• Process analysis
• Quantitative quality plans
(Intuitive)
• Process dependent on individuals
• Established basic project controls
• Strength in doing similar work, but faces major risk when
presented with new challenges
• Lacks orderly framework for improvement
• Training
• Technical Practices (reviews,
testing)
• Process focus (standards,
process groups)
(Ad hoc/chaotic process)
• No formal procedures, cost estimates, project plan
• No management mechanism to ensure procedures are
followed, tools not well integrated, and change control is lax
• Senior management does not understand the issues
• Project Management
• Project Planning
• Configuration Management
• Software Quality Assurance
• Requirements Management
Risk
Shrinking the Software Scrap Heap reserved
All rights

3. Software Quality Assurance
• The purpose of Software Quality Assurance is to
provide management with appropriate visibility
into the process being used by the software project
and of the products being built. (CMU/SEI-93TR-025, Key Practices of the Capability Maturity
Model, Version 1.1, Mark C. Paulk et al.)
• SQA is forward looking and predictive.
• SQA is not testing.
Shrinking the Software Scrap Heap reserved
All rights

4. SQA is NOT Testing!
Sr. QA Engineer
Position Type Full-Time Employee
Company Name Edgelink
Location Portland, OR
Salary $50K -$75K
Experience 10-15 Years Experience
Desired Education Level Bachelor of Science
Recruiting Firm
- 3+ years of experience in Software Quality Assurance and testing.
- QA testing experience as a part of a software testing team in a Microsoft development environment.
- 2 years experience testing with browsers.
- Working knowledge of software testing disciplines and methodology (e.g. negative, white/black box, performance
and stress testing techniques).
- Excellent analytical skills and commitment to continuous testing improvement.
- Excellent verbal and written communication skills.
- Experience in evaluating and using configuration management tools, defect tracking tools, debugging tools, and
testing tools.
- Expert knowledge of Test Planning, Test Case Specification, and Test Procedure Development.
- Test Automation expertise with leading Test Tool environments (Silk, Winrunner, Mercury tools, etc).
Shrinking the Software Scrap Heap reserved
All rights

5. Another Example
- Is software quality assurance your career focus?
- Do you thrive on building world-class software quality assurance systems and
processes that ensure the delivery of perfect applications?
- Do you feel at home in a dynamic, fast-paced, time-critical environment?
- Are you a perfectionist with a high degree of attention to detail?
- Do you take a logical, systematic, yet creative, approach to the design,
development, and implementation of systems and processes?
-The Gallup Organization is seeking a talented Software Quality Assurance Specialist who will
design, develop, and implement software quality assurance systems and procedures that will support
the development and delivery of perfect software applications. Software Quality Assurance
Specialists will work in our Omaha, Nebraska, location.
Responsibilities for this position include analyzing and testing project deliverables throughout the
development life cycle to ensure that business and technical requirements are being met. This may
include, but may not be limited to: traceability, defect identification, performance, capacity, security,
and recovery. Software Quality Assurance Specialists also partner with our development,
management, and technical staff to provide ongoing assistance and education of our quality
assurance testing methodologies, systems, processes, and standards.
Shrinking the Software Scrap Heap reserved
All rights

6. What SQA does….
•
•
•
•
•
•
•
Reviews project data (metrics)
Reviews project work practices
Analyses project work products
Analyses verification data
Makes projections – risk analysis
Estimates project quality factors (cost, reliability, etc.)
Reports results to management – Provides Assurance
(or a warning)
• Escalates problems to a level where they will be
solved.
Shrinking the Software Scrap Heap reserved
All rights

7. One more try!
Software Quality Engineer
Job Requirements:
Perform Software Quality Assurance activities as a key member of a team developing software for precision
control systems used on aircraft, launch vehicles, missiles, defence systems, satellites and space vehicles. The
successful candidate will be responsible for monitoring software development process execution to ensure
effective compliance with internal and external requirements. Software Quality Engineering activities include:
•Auditing software process activities
•Evaluating software work products to established standards
•Ensuring root cause analysis and preventative/corrective action
•Collecting and analyzing process metrics
•Recommending and developing software process improvements
The position requires excellent judgment, interpersonal, communication and organizational skills. The successful
candidate must be familiar with standard software process models, software life cycle activities, audit techniques
and process improvement. Education required: four-year degree (engineering or computer science discipline
desired) with a minimum one-year experience in a software development environment.
Shrinking the Software Scrap Heap reserved
All rights

8. SEI CMM Summary
Level
5
Optimizing
4
Managed
3
Defined
2
Repeatable
1
Initial
Characteristic
Characteristics
Key Challenges
Result
s
• Improvement fed back into process
Productivity
• Data gathering is automated and used to identify weakest
process elements
&
• Still human- intensive process
• Numerical evidence used to justify application of technology • Maintain Organization at
Quality
to critical tasks
optimizing level
• Rigorous defect-cause analysis and defect prevention
(Quantitative)
• Measured Process
• Minimum set of quality and productivity measures
established
• Process database established with resources to analyze its
data and maintain it
(Qualitative)
• Process defined and institutionalized
• Software Engineering Process Group Established to lead
process improvement
• Changing technology
• Problem analysis
• Problem prevention
• Process measurement
• Process analysis
• Quantitative quality plans
(Intuitive)
• Process dependent on individuals
• Established basic project controls
• Strength in doing similar work, but faces major risk when
presented with new challenges
• Lacks orderly framework for improvement
• Training
• Technical Practices (reviews,
testing)
• Process focus (standards,
process groups)
(Ad hoc/chaotic process)
• No formal procedures, cost estimates, project plan
• No management mechanism to ensure procedures are
followed, tools not well integrated, and change control is lax
• Senior management does not understand the issues
• Project Management
• Project Planning
• Configuration Management
• Software Quality Assurance
• Requirements Management
Risk
Shrinking the Software Scrap Heap reserved
All rights