2. Contents
Chapter 1 Overview of Quality Auditing 1
Chapter 2 Audit Definitions and Applications 9
Chapter 3 Phase 1--Audit Initiation and
Preparation 26
Chapter 4 Phase 2--Performance of an Audit:
On-Site Evaluation 40
Chapter 5 Phases 3 & 4--Reporting and Closure
Phases 55
Chapter 6 Establishing the Framework for an
Audit Program 64
Chapter 7 The First Seven Elements of a Quality
System 69
Chapter 8 The Second Seven Elements of a
Quality System 78
Chapter 9 The Final Seven Elements of a
Quality System 87
Chapter 10 Statistical Process Control 96
Chapter 11 Statistical Sampling for Auditing 102
4. Audit Overview
• Auditing is an art and a science.
• Audits are snapshots in time.
• Audits should not be
confrontations.
• Auditors investigate, verify, and
confirm.
• The audit should be used as a
tool for continual improvement
of the quality management
system.
2
5. Auditor Knowledge
• Quality management system
requirements
• Product specifications
• Process procedures
• Investigation techniques
• Sampling techniques
• Communication techniques
3
7. Two Types of Quality
Standards
• System standards
– Examples
• ANSI/ISO/ASQ Q9000 standards
• QS-9000 automotive standards
• ISO 14000 environmental standards
• Product standards
(specifications)
– Examples
• Chemical and physical property
requirements
• Dimensional tolerances
5
8. Review and Verify
Documentation,
Implementation, and
Effectiveness of
• Policy manuals
• Procedure manuals
• Work instructions, drawings,
etc.
• General quality management
system requirements
• Individual product
specifications
6
9. Audits Can
• Enhance internal quality
management systems
• Be part of continual quality
improvement efforts
• Help establish trusting
relationships internally and
externally
• Find problems before someone
else does!
7
10. Audits Are Not
• Acceptance of product
• Punch lists
• Public hangings
• Fault Finding
• Ill-prepared
• Antagonistic
• Biased
• Rambling
8
12. What Is an Audit?
• ANSI/ISO/ASQC Q10011-1-
1994: “A systematic and
independent examination to
determine whether quality
activities and related results
comply with planned
arrangements and whether these
arrangements are implemented
effectively and are suitable to
achieve objectives.”
• Quality Applications:
– System audit
– Process audit
– Product audit 10
I
II
III
13. Institute of Internal
Auditors
• Definition from IIA
– “Internal auditing is an
independent, objective assurance
and consulting activity designed
to add value and improve an
organization’s operations. It helps
an organization accomplish its
objectives by bringing a
systematic, disciplined approach
to evaluate and improve the
effectiveness of risk management,
control and governance
processes.”
11
14. Additional Audit Terms
• Compliance audit
– Compliance to standards
– Review of processes
– Review of data
• Operational audit
– Time studies
– Work studies
• Readiness review by internal
management
– ANSI/ISO/ASQ Q9000 system
• Management review
– Processes, procedures, & controls
• Survey
– Comprehensive evaluation 12
15. System Audit Definition
“ A documented activity
performed to verify, by
examination and evaluation of
objective evidence, that
applicable elements of the
quality management system are
appropriate and have been
developed, documented, and
effectively implemented in
accordance and in conjunction
with specified requirements.”
13
17. Types of Audits
(continued)
• Survey
– Comprehensive evaluation of
facilities, resources, economic
stability, personnel, capability
• System audits
– Verification that a quality
management system exists and is
being followed
• Process audits
– Verifies procedures exist, correct, &
are being followed under standard,
rushed/adverse conditions
• Product audits
– Verifies inspector capabilities and
correct use of specifications 15
19. Characteristics of Audits
• Are independent of doer
– Performed by a person not
responsible for production or
services in area being audited
• Have measurement criteria
– Specifications (quality
management system and product
requirements)
– Attribute data (good/bad)
– Variables data (actual
measurements)
17
20. Characteristics of Audits
(continued)
• Are performed by competent
personnel
– Trained audit team members
• Are cost effective
– Prevention of problems
• Are samples of the system
– Isolate one small moment in time.
– Represent the total population
• Must be sure the sampling
procedure is correct.
– Avoid making incorrect
assumptions 18
21. Two Types of Sample
Process Errors
• Type I error (alpha error)
– We assume something is
unacceptable when it is actually
acceptable
• Producer’s Risk
• Type II error (beta error)
– We assume something is
acceptable when it is actually
unacceptable
• Consumer’s Risk
• Sample consideration
– Size, amount, location, & errors
19
22. Use of Audits
• Continual improvement efforts
– Internal and external
• Benchmarking
• Value-added suppliers
• Certification
– Internal (quality management
system)
– External (suppliers)
Note: Do not use audits alone to
determine acceptance of
product.
20
23. Why Audit?
• Competition
– External--We want low-cost, high-
value suppliers
– Internal--We want to verify our
systems are adequate/effective
• Regulation
– Imposed by customers
– Self-imposed
– General quality standards or
internal quality standards
• Self-preservation to audit
ourselves before someone else
does
– OSHA 21
24. Audits Improve
Performance if They
• Meet management needs
– Provide feedback on areas
needing improvement
• Provide a forward look in time
– Can predict where process
controls or product measurements
are needed
• Measure effectiveness and
compliance to internal and
external policy and contract
requirements
22
25. Who Is the Customer?
• There are three customers
(clients) we must satisfy on
each audit.
– Auditee
• Person being audited (internal or
external)
– Boss or manager
• Give authority to perform internal
audit
– Buyer of the service (client)
• Buyer of service on external audit
23
26. What Managers Do
• Responsible for entire system
and its various processes
• Continual process improvement
– Plan
– Do
– Check (Study)
• audits address the “check” function
of the cycle by checking and
monitoring a system
– Act
Plan
DoCheck
Act
24
27. Audit Administration
• Formal audit procedures
– Initiation and preparation 25%
• Areas to be audited
• Team composition and training
• Notification of auditee
• Checklist development
• Transportation
– On-site audit performance 50%
– Reporting and closure 25%
• Audit report
• Audit records
• Tracking and closure
25
29. Audit Initiation Phase
• Audit initiated by client
– Develop audit plan
• Lead auditor (audit manager) and
client (approves)
• Time allocated for audit
– Client
• Determine need for audit
• The purpose of audit
• The type of audit
• The objectives of audit
– Lead auditor
• Scope of the audit
27
30. The Preparation Phase
• Items to be covered in the audit
plan, which must be defined:
– Objective [purpose, manuals,
procedures, work instructions]
– Scope [facility, product, process,
system elements, departments]
– Resources [trained auditors,
technical expertise, team size]
• Length of audit =
< 500 employees: 1-3 days
500-1000 employees: 4-5 days
– Initial contacts
– Checklists
– Past history 28
31. Lead Auditor
Responsibilities
• Prepare the audit plan
– 50% of work
• Select the audit team
– Based on plan
• Brief the audit team
– Safety issues
– Brief auditee
• Submit a report
– 2 weeks maximum
29
34. Auditor Musts
• Knowledge of:
– Verification techniques
• Show me…
• Objective evidence
– Standards
– Sampling techniques
– Human nature
• Four faces of Frank
– Time management
• You are in control
32
36. Contact the Auditee
• Informal contact
– Date
– Purpose
– Scope
• Formal contact by letter/memo
– Time and date of audit
– Purpose
– Scope
– Activities to be audited
– Interfacing organizations
– Applicable quality requirements
– Identification of team members
– Preliminary schedule 34
37. Perform Desk Audit
• Review quality system
documentation of auditee
– Policy Manual/Procedures
• Determine what control systems
are in place
– What facts are available?
• What is the product?
– Become familiar with product
• Review prior audits
• Investigate exemptions or
waivers
• Review performance history 35
38. Develop Checklists that
Provide:
• Structure
– Define before audit/use team
• Required coverage
– Not designed to stifle creativity.
• A communication document
– A prompt to your next question.
• A place to record data
– Objective evidence is required for
anything less than satisfactory.
• Help in time management
– Pace of the audit
36
39. Auditors--Points to
Consider
• Auditors can use:
– Canned checklists
– Yes/no checklists [attributes]
– Rating checklists [variables]
• Things to consider when
developing checklists:
– What is process during normal
operations?
• Rushed conditions?
• When there is a problem?
37
40. Summary of the
Preparation Phase
• Define the audit objective
• Establish the audit scope
• Allocate resources
• Contact the auditee
• Develop checklists
• Review history
• Understand the process and
control systems
38
41. Results of the
Preparation Phase
• Audit plan
– auditee, purpose, scope, activities
to be audited, team, standards
• Checklist questions
• Items and reference location of
each requirement
• Initial evaluation
– Based on documentation received
• Desk audit
• Past audit results
• Plan for action
– Areas needing attention/audited 39
43. The Performance Phase
• Hold opening meeting
• Conduct on-site audit
• Review controls of quality
management system
• Verify that the system is
working
– Is it documented, implemented,
and effective?
• Share information
41
44. The Opening Meeting
• Must have:
– Entire audit team present
– Auditee’s plant manager/staff
• Requires two-way
communication
– Trade introductions
– Restate objectives
– Form initial impressions
– State any areas of concern
– Settle logistics; space, lunch, etc.
• Should take no more than 30
minutes 42
45. The Opening Meeting
[continued]
• If needed:
– Allow 15 minute maximum for
auditee to present quality
management system
enhancements
• Distribute the checklist(s)
• Define the audit schedule
• Be on time
43
46. On-Site Evaluation
• Auditors can be compared to
florists:
– Collects evidence [flowers]
– Puts it on worksheets [baskets]
– Arranges the data [flower
arrangement]
– Delivers the report [delivers
flowers in baskets]
44
47. Causes of Quality Issues
• Lack of top management
support
• Lack of organization
• Lack of training
• Lack of discipline
• Lack of resources
• Lack of time
• Lack of teamwork
• Lack of knowledge
• Lack of consistency
– Look for symptoms of these
issues 45
48. Evaluating Information
• Formal [documented] control
systems vs. Informal
[understood/verbal] control
systems
– Does top management believe in
quality and are these beliefs
communicated throughout the
organization?
– Is the control system used by
management?
– Is the control system adequate?
• Are key variables identified?
– Is the control system working?
• How is it monitored? 46
49. Verification Methods
• Three primary methods used:
– Tracing
– Corroboration
– Sampling [discussed in Chapter
11]
A Contract Random
Horizontal
Vertical
47
50. Tracing Techniques
• Techniques of tracing
– Contract -- Series of audits
performed on a contract as the
contract progresses [2nd party]
– Random -- Review what is taking
place at time of audit [3rd party]
– Horizontal
• Backward -- From the end of
process [1st, 2nd, 3rd party]
• Forward -- From the beginning of
process [1st, 2nd, 3rd party]
• Middle -- Backward or forward
from some critical point in process
[1st, 2nd, 3rd party]
48
51. Tracing Techniques
[continued]
• Vertical -- Reviews management
layers for elements such as quality
goals and quality communications
• Departmental -- Reviews numerous
quality elements within a
department
• Element -- Reviews a quality
element in various departments
49
52. Corroboration
• A statement is not a fact until it
is corroborated by someone else
or is verified by a document.
– The facts must agree:
• Two different auditors
• Two different records
• Two different interviews
– Combination of the above
• Tie conclusions to what is tangible
– Qualitative [some documents reviewed
did not have…]
– Quantitative [out of 50 documents
reviewed, 5 were found to have…]
• Forms, records, procedures
50
53. Sources of Data Collection
• Physical evidence
– Reinspect, retest, recheck
– Documentation and records
• Sensory observations
– Rusty parts
– Nonconforming parts stored with
good parts
– Observation of tasks being
performed
• Comparisons and relationships
– Patterns and trends
• Interviews and questions
– Valuable source of information
– No place for sexism or biases 51
54. Effective Interviews
1.Put person at ease.
– Your presence is threatening.
2. Explain your purpose.
– Demonstrate competence.
3. Use proper questioning
techniques.
– Ask open-ended questions.
– Use “pregnant pause”.
– Ask why five times/five w’s.
4. Analyze and verify what is said.
– Verify a claim.
– Write notes out loud/no secrets.
5. Explain your next step.
– Conclude cordially. 52
55. Brief the Auditee During
the Audit
• Short and to the point
– Discuss verifications and
concerns
– About 5-10 minutes
– In each area as you progress
through the facility
53
56. Auditor Team Meetings
• Informal
– End of each day with audit team
• Auditor sharing
– Facts, conclusions, problems
• Replanning
– Changes to the audit schedule
• Reporting
– Preliminary audit results
• Status of the audit or areas
covered
– Areas completed, areas to be
examined, concerns & issues 54
58. Reporting Phase
• Informal report
– Exit meeting--60 minutes
• Formal report
– Written & distributed within two
weeks after the audit
“Clear and precise reports”
• Statements of noncompliance
contain
– Requirement
– Location
– Evidence
– Finding(s) 56
59. The Exit Meeting
• Same people as opening
meeting
• Lead Auditor presents summary
• Disclaimer about sampling
• Lead auditor reads findings
• Explain follow-up and
corrective action response
process.
57
60. Findings
• Observation
– Write what you were doing.
• “While auditing Purchasing I found
drawings for new suppliers were not
being marked on form…”
• Attribution
– Restate system’s deficiencies
relating to the standard or written
documentation of company.
• “This is a finding against Procedure
P01-01 which requires all new
drawings sent to suppliers…”
• Explanation
– Explain why this is a finding
• “The drawing must be recorded to
ensure supplier has latest version.”
58
61. Degree of Severity of
Findings
• The report should order findings
by their importance.
– Major finding--conflict with
contractual or procedural
requirements. [6 maximum]
– Minor finding--Isolated
observations in conflict with
contractual or procedural
requirements. [6 maximum]
– Concerns--Observations that may
not be part of the required quality
management system, but are
worthy of further consideration.
59
62. Content of the Formal
Audit Report
• Formal audit report from audit
team to client should include:
– Purpose and scope of audit
– Participant’s names
– Background information
– Summary of the results
– Identified weaknesses in order of
importance
– Date and signature of lead auditor
60
63. The Closure Phase
• Auditee has 30-45 days to
respond with plan of action.
• Auditee’s corrective action
evaluated.
• Formally close corrected items.
• Items requiring on-site
verification are reaudited.
• Letter of closure issued.
• All documentation filed.
61
64. Retention of Audit
Documents
• “Working papers” should be
maintained for at least one audit
cycle for follow-up audits.
• Follow the required Quality
Record Procedure requirements.
62
65. Pitfalls to Avoid
• Inadequate planning/preparation
• Inadequate communication
prior to audit
• Lack of clearly defined scope
• Inadequate knowledge of
standards
• Improperly trained auditors
• Failure to reevaluate
implemented corrective actions
63
67. Establishing the
Framework for an Audit
Program
• Top management gives authority
to establish audit function
– Policy Q17-01
– Procedure P17-01
• Audit administrator is assigned
– Management Representative
• Purpose and scope established
– Minimum compliance purposes
– Continuous quality purpose
65
68. General Principles to
Administering an Audit
• Objectivity
– Independence
– Attack problem, not people
– Avoid conflict
– Verify data
• Confidentiality
– Information should not be shared
by auditor from company to
company
66
69. Audit Staff
• Auditors are selected on the
basis of:
– Experience
– Knowledge
– Communication skills
– Personal traits
– Judgement
67
70. Audit Records Retention
• Permanent records
– Notification letter
– Audit plan
– Pre-assessment questionnaire
– On-site audit/evaluation
– Working papers
– Audit report letter
– Closure letter
– Auditor training/qualification
• Nonpermanent records
– Notes, supporting documents
• Audit records kept 3 years 68
72. The First Seven
Elements of a Quality
Management System
1. Management responsibility
2. Quality system
3. Contract review
4. Design control
5. Document control
6. Purchasing
7. Customer-supplied product
70
74. 2--Quality System
[checklist]
• General requirements
• Documentation requirements
• Quality Manual
• Quality management system
planning
I quality manual
II procedures
III work instructions
IV Forms, tags, labels
72
75. 3--Contract Review
[checklist]
• Customer focus
– Interested parties
• Customer-related processes
– Determine mutually acceptable
processes
• Review of requirements related
to the product
• Customer communication
– Product information
– Amendments
– Customer complaints/feedback
73
76. 4--Design Control
[checklist]
• Design and development
– Planning
– Inputs
– Outputs
– Review
– Verification
– Validation
• Control of design and
development changes
74
77. 5--Document & Data
Control [checklist]
• Control of documents
– Approval
– Review and update
– Current revision status
– Latest versions at points of use
– Documents remain legible and
retrievable
– External documents are controlled
– Obsolete documents are identified
75
79. 7--Control of Customer
Supplied Product
[checklist]
• Customer property
– Ingredients or components
– Packaging materials
– Customer material in storage
– Transport to a third party
– Customer intellectual property
77
81. The Second Seven
Elements of a Quality
Management System
8. Product identification and
traceability
9. Process control
10. Inspection and testing
11. Inspection, measuring, and
test equipment
12. Inspection and test status
13. Control of nonconforming
product
14. Corrective & preventive action
79
82. 8--Product Identification
and Traceability
[checklist]
• Identification and traceability
– Drawings
– Documents
– Bar codes
– Computer
– Tagging
– Labeling
– Handling/storage
• Segregation/quality status
• Quantities
• Records of traceability
• Shipment records 80
83. 9--Process Control
[checklist]
• Product realization
– Planning of product realization
• Production and service
provision
– Control of production and service
provision
– Does not include servicing
– Validation of special processes
• Monitoring and measurement of
processes
– Reaction time
– Yield 81
84. 10--Inspection and
Testing [checklist]
• Monitoring and measurement of
product [from receiving
inspection to product delivery]
– Receiving inspection and testing
• Lot sampling plans
• Skip lot
• Certifications
– In-process inspection and testing
• First piece setup
• Process audits
– Final inspection and test
• 100% inspection
• Lot sampling
• Product audits
82
85. 11--Inspection,
Measuring, and Test
Equipment [checklist]
• Control of measuring and
monitoring devices
– Calibration procedures
– Calibrated at defined intervals
– Identified and safeguarded
– Protected from damage
– Personnel trained
– Maintain records
– Control environment
– Confirm computer software
83
86. 12--Inspection and Test
Status [checklist]
• Identification and traceability
– Status of material maintained
throughout process
• Paper records
• Computer records
• Product tags
• Bar codes
• Traceability of product with releases
– Identification of responsible
personnel for conformance
decisions
• Records of product release
84
87. 13--Control of
Nonconforming Product
[checklist]
• Identification [NCMR]
– Marking, tagging, computer
• Segregation
– Physical, records, computer
• Disposition
– Evaluation by authorized personnel
• Accept with repair by concession
• Accept without repair by concession
• Rework to specifications
• Return to supplier [vendor]
• Regraded for alternate application
• Scrap
85
88. 14--Corrective and
Preventive action
[checklist]
• Identification of problem
– Causes
• Common or special
• Repeat or isolated
• Root cause determination
• Evaluation of impact
– Costs
– Performance
– Safety
– Customer satisfaction
• Implementation of corrective
actions [prevent recurrence]
• Implementation of preventive
actions [prevent occurrence] 86
90. The Final Seven
Elements of a Quality
Management System
15. Handling, storage, packaging,
preservation, and delivery
16. Quality records
17. Internal quality audits
18. Training of personnel
19. Servicing
20. Statistical techniques
21. Quality cost analysis
88
91. 15--Handling, Storage,
Packaging, Preservation,
and Delivery
• Handling
– Prevent damage/deterioration
– Maintain identification
• Storage
– Maintain identification/traceability
• Packaging
– Prevent damage/maintain identification
• Preservation
– Protect from deterioration
• Delivery
– Protection of quality after inspection
• Feedback system 89
92. 16--Control of Quality
Records
• Examples of quality records:
– Management review
– Employee education, skills,
training, and experience
– Production criteria
– Design records
– Supplier evaluations and actions
– Special process validations
– Customer property damage
– Calibration and verification
– Internal audits
– Authorizing release of product
– Nonconforming product
– Corrective/preventive action 90
93. 17--Internal Quality
Audits
• Scheduling
– Frequency, dates, time involved
• Objective
– Type of audits
• Scope
– Location, element, audit team
• Procedures
– Training program, checklists
• Implementation
– On-site evaluation
• Reporting
– Corrective actions
• Follow-up 91
94. 18--Training of
Personnel
• Training needs planned/identified
– Certifications for specific employees
• Training is provided
– Competency based on qualifications
• Employee training is verified
– Qualification/certification on the
basis of;
• Education, training, experience, and
evaluation of skills verified
• Human factors are considered
– Inspire continuous improvement
– Be knowledgeable about quality
– Provide a good work environment
92
95. 19--Servicing
• Field servicing of products or
services to meet requirements
– Performed to written procedures
• [P19-01]
• Liability
– Warranty
• Claims/traceability
– Ability to recall
• Records, designs, specifications,
process controls, testing results
• Safety
– Risks
• [MSDS] 93
96. 20--Statistical and
Analytical Techniques
• Process control techniques
– Flowcharts
– Control charts
– Trend charts
– Histograms
– Pareto charts
– Cause/effect diagram
– Inspection plans
• Assessment tools for applications
– Benchmarking
– FMEA
– Reliability predictions 94
97. 21--Quality Cost
Analysis
• Standard cost system
– Standard cost vs actual cost
• Cost of nonconformance
– Value lost to company
• Quality cost system
– Prevention--Blue $
• QIP process
– Appraisal--Blue $
• Inspection, calibration, testing
– Internal failure--Red $
• Scrap, rework
– External failure--Red $
• Returns, warranty claims
95
99. Key Process and Product
Variables
• Process map [flowchart]
– Identification of:
• Quality control points
• Location of activities
• Who is responsible for activities
– Comparison to external and
internal customer requirements
[quality plan]
– Process and inspection data
feedback to control system
Start or end
Go/no go
Task
97
100. Variability of Processes
and Products
• Causes of variability in
processes
– Man, machine, material, methods,
mother nature
• Evaluation of the causes of
variability
– Special causes
– Common causes
Process
MethodMaterial
MachineMan
Mother
Nature
98
101. Statistical Control charts
and Process Stability
• Statistical limits [based on past
data]
– Minimum 20 samples
• Statistical control
– Common cause variation
• Data points outside limits
– Out of control
• Data points exhibiting patterns
– Runs, trends, 7 consecutive
• Determination of special causes
or process changes
– Notes 99
102. Types of Data Collection
• Variables Control Charts
– Measurement
• X-bar, R control chart
– Samples are averaged, range plotted
• Individual, moving range [Xmr]
– One unit measured, range plotted
• Attributes Control Charts
– Count
• Nonconforming products
– P charts
– NP charts
• Nonconformities per unit
– C charts
– U charts
100
103. Signals vs Noise
• Out of control signals or
patterns
– Data points beyond the control
limits
– Data points in a run [7 or more
points above or below central
line]
– Data points in a trend [7 or more
points going up or down and
crossing the central line]
– Data points in a periodic
configuration
– Data points stratified
101
105. Aspects of Sampling
• Sampling obtains information
from a part of the total
population or field
• Sampling is used in:
– Accounting
– Business
– Industry
– Quality auditing
102
106. Methods Used Must Suit
Needs of the Audit
• Types of approaches
– Estimation sampling
• Provides the auditor with an estimate of
the frequency of occurrences of an event
in the field and considers sampling error
• Desired sample precision is based on risk
and consequences of making an error
– Inventory accuracy vs airplane engine defects
– Discovery sampling
• It is used to find one nonconformance
within a given field size
– Acceptance sampling
• Is based on a continuous process and is
not suggested for use in quality audits
103
107. Sample Selection
• Establish what is to be sampled
depending on the scope of the
audit
– Critical work instructions
– Highly stressed, high-volume,
high precision areas
– Contributory factors that make the
process successful
– Time allocated
104
108. How to Select an
Unbiased Sample
• Equal chance of selection
• After process is running
consistently
• Selection of predetermined
number of random samples
– Use of random number tables
– Use of calculator
– Use of a book
105
Deck of cards to show
biased sample
