3. Software Quality Assurance
โข What is โqualityโ?
โข IEEE Glossary: Degree to which a system,
component, or process meets (1) specified
requirements, and (2) customer or user needs
or expectations
4. Software Quality Assurance
โข What is โqualityโ?
โข IEEE Glossary: Degree to which a system,
component, or process meets (1) specified
requirements, and (2) customer or user needs
or expectations
โข ISO: the totality of features and
characteristics of a product or service that
bear on its ability to satisfy specified or
implied needs
5. Software Quality Assurance
โข An alternate view of Quality:
โ is not absolute
โ is multidimensional, can be difficult to quantify
โ has aspects that are not easy to measure
โ assessment is subject to constraints (e.g., cost)
โ is about acceptable compromises
โ criteria are not independent, can conflict
7. What is Software Quality Assurance
(SQA)?
โข โSet of systematic activities providing
evidence of the ability of the software
process to produce a software product that
is fit to useโ
โ G. Schulmeyer and J. McManus, Software
Quality Handbook, Prentice Hall, 1998.
8. What is SQA?
โข Monitoring processes and products throughout
the software development lifecycle to ensure
the quality of the delivered product(s)
โข Monitoring the processes
โ Provides management with objective feedback
regarding process compliance to approved plans,
procedures, standards, and analyses
โข Monitoring the products
โ Focus on the quality of product within each phase
of the SDLC
โข e.g., requirements, test plan, architecture, etc.
โ Objective: identify and remove defects throughout
the lifecycle, as early as possible
9. Quality of Software developed in-house
& COTS components
โข SQA processes apply when integrating
purchased or customer-supplied software
products into the developed product
โข Question. How do you determine the
โqualityโ of COTS components?
โ Current research problem
10. Process Assessment
โข Use of standards and process models has a positive
impact on the quality of the software product
โ Disciplined, controlled development process
โข Examples include:
โ ISO 9001
โ CMM
โข CMU SEI, 5 levels
โ SPICE
โข Developing a standard for software process assessment
โข ISO joint committee, Europe, Australia
โ IEEE 1074, IEEE 12207, โฆ
12. Product Assessment
โข Reviews, inspections, walkthroughs of
Plans, reports, models, standards
โ Project management, quality assurance,
training, test plan(s)
โ Requirements, analysis, architecture, detailed
design model, test cases
โ Issue or problem reports
โ Metric reports
โ Traceability reports
โ Documentation, coding standards
โ โฆ
13. Software Reviews
โข They may include managerial reviews, acquirer-supplier reviews,
technical reviews, inspections, walkthroughs, and audits.
โข Inspection:
โ A formal evaluation technique in which an artifact (e.g., software
requirements, design, or code) is examined in detail by a person or group
other than the originator
โ detect faults, violations of development standards, and other problems.
โ review members are peers (equals) of the designer or programmer.
โ data is collected during inspections for later analysis and to assist in future
inspections.
Note
โ Introduced by Fagan, 1976.
โ Fagan, M., โDesign and Code Inspections to Reduce Errors in Program
Developmentโ, IBM Systems Journal, 15, 3 (1976), pp. 182-211
โ Fagan, M., โAdvances in Software Inspectionsโ, IEEE Transactions on Software
Engineering, 12, 7(July 1986), pp. 744-751
15. Defect Checklists
โข Useful to support reviews, inspections, walkthroughs
โข Expertise is captured in a list format
โ Less experienced people can use
โข Straightforward to use (each check should be clear, simple to assess/apply)
โ Improve consistency of assessments
โข Example architecture checklist used in undergrad./grad.
courses for OO
โ spreadsheet in in the course materials subdirectory
โข One or more architectural styles are selected.
โข Capabilities and interfaces are defined for subsystems.
โข Capabilities of and interfaces among subsystems support all of the use cases.
โข Concurrency defined.
โข Distribution defined.
โข Error handling defined.
โข Start up and shut down defined.
โข Data persistency defined.
โข Rationale for the model is provided.
โข Other
16. Verifying Formal Specifications
โข Formal specifications may be verified in a number of
different ways:
โSyntax, typechecking
โข If the notation is typed
โSimulated
โModel checked (e.g., SPIN)
โProven correct (e.g., HOL, PVS)
โข More straightforward? Less assurance of correctness; fully
automated
โข Less straightforward? Higher assurance of correctness; not
fully automated
More
straightforward
Less
straightforward
17. Problem Reporting, Tracking, and Resolving
โข Describe the practices and procedures to be followed
for reporting, tracking, and resolving problems
โ Who can report a problem?
โ How is it reported?
โ How is is tracked?
โ Who determines if it is a problem that going to be resolved?
โ How is it assigned for resolution?
โ How does the person indicate it has been corrected?
โ Who reviews it to determine if it can be closed?
โข Problems can be product or process related
โ e.g. incorrect requirement, incomplete class definition, code
defect, ambiguous description in user documentation,
process to review detailed design is not clearly defined, etc.
18. Metrics
โข Metrics for each artifact
โข e.g., Requirements
โ Number of requirements
โ Number of changes per requirement
โข Called โchurnโ rate
โ Characterization of defects
โข Not testable, ambiguous, inconsistent, incorrect,
incomplete redundant, infeasible, โฆ
โข Major or minor defect
โข Phase defect detected
โข Cost to fix
19. Tools, techniques, training
โข What tools?
โ e.g., CVS for CM, excel spreadsheet for
problem reporting/tracking, ...
โข What techniques?
โ e.g., formal peer review for deliverables,
checklists for defect detection, ...
โข What training is needed on tools,
techniques?
20. Media Control
โข Identify the media for each intermediate and
deliverable artifact
โข Documentation required to store the media, including
the backup and restore process
โข Protect computer program physical media from:
โ unauthorized access
โ inadvertent damage
โ degradation
21. Architecture Analysis Methods
โข Why evaluate an architecture?
http://www.slideshare.net/kevinjew/evaluating-software-
architectures-presentation
โข Specialized techniques available:
http://www.slideshare.net/timmenzies/architecture-
tradeoff-analysis-method-presentation
SEI presentation and technical report on ATAM are in the
course subdirectory