J.Gras Six SigmaTSIC 2010

Six Sigma based quality control
- Local and national perspectives -
Jérémie Gras
Médecin Biologiste
Laboratoire - Clinique St-Luc Bouge
Turning Science Into Caring Conference
Living Tomorrow – Vilvoorde
7th October 2010

- PLAN -
1) Six Sigma basics
2) Internal Quality Control applications- the local
perspective
3) External Quality control applications- the national
perspective
4) Tendencies and controversies in the use of Six
Sigma in clinical laboratories
Jérémie Gras - Six Sigma based quality control, local and national perspectives – TSIC conference 2010
Clinique St-Luc Bouge

1. Six Sigma Basics
• Six Sigma is a global management strategy introduced by
Motorola in the 80’s
• First application of Six Sigma occured in the production
process
• Most of Fortune 500 companies, especially manufacturing
companies, have implemented Six Sigma with tremendous
success in terms of customer satisfation and global
profitability

1. Six Sigma Basics
But what is Six Sigma exactly ?

1. Six Sigma Basics
A few definitions of Six Sigma:
• Six Sigma is a problem solving methodology
• Six Sigma performance is a statistical term for a process
that produces fewer than 3.4 defects per million
opportunities (3.4 DPMO or 3.4 ppm)
• A Six Sigma organisation uses Six Sigma to improve
performance: continuously lower costs, grow revenue,
improve customer satisfaction, reduce complexity, lower
cycle times, minimize errors

1. Six Sigma Basics
The Six Sigma methodology
• Define, Measure, Analyse, Improve, Control
• Breaktrough methodology: DMAIC
1) Define= define problems
2) Measure= measure performance and determine error rate
3) Analyse= analyse data to find the cause of errors
4) Improve= improve the process, reduce errors
5) Control= control the improved process

1. Six Sigma Basics
The Six Sigma methodology: DMAIC
• Seems simple, but terribly effective
• Measure is the key step: careful documentation with
numerical data helps a lot to make decisions !
• Six Sigma introduces a new vision of quality: quantitative
quality that is clearly measurable !

1. Six Sigma Basics
But how can we measure quality ?
• Six Sigma is a management methodology
• But it is also a scale- the Sigma scale
• That scale allows to quantitate quality for most human
activities on an industrial scale

1. Six Sigma Basics
The Sigma scale (1/3)
• Six Sigma principles were firstly introduced in the industrial
production area, aiming at creating products with a minimal
number of defects
• Six Sigma as a value corresponds to 3,4 defects per million
opportunities
• 3,4 DPMO or 3,4 ppm represents world class quality

1. Six Sigma Basics
Sigma Errors (%) Errors (ppm)
1 69 % 691462
2 31 % 308538
3 6.7 % 66807
4 0.62 % 6210
5 0.023 % 233
6 0.00034 % 3.4
7 0.0000019 % 0.019

1. Six Sigma Basics
• a Sigma value indicates how much errors are occurring: a
high Sigma level correlates with a process with a low
number of defects
• a Six Sigma process has so little variation, that even a
variation of 6 standard deviations will fit in the tolerance
limit for the process
That is World Class Quality
That is where the term “Six Sigma” comes from

1. Six Sigma Basics
Six Sigma applications in clinical laboratories
• In 2010, we can state that Six Sigma has been implemented
and deployed in many sectors of In Vitro Diagnostics:
Firstly with IVD manufacturing companies
– Many constructors have implemented Six Sigma in
manufacturing and other areas (pricing, etc): Abbott,
Beckman Coulter,...
– Some provide special consulting services to improve
lab organisation: Valumetrix, also Abbott,…

1. Six Sigma Basics
• In 2010, we can state that Six Sigma has been implemented
and deployed in many sectors of In Vitro Diagnostics:
Secondly in clinical laboratories, with two applications:
1) Application of the Six Sigma management methodology
(DMAIC) to improve global laboratory performance
2) Quantification of local analytical performance on
the Sigma Scale and QC rules selection

1. Six Sigma Basics
• Successfull exemples of DMAIC applications in clinical
laboratories*:
• To reduce errors in the data entering department
(Riebling 2004)
• To reduce TAT by taking action on the pneumatic tube
system (Simmons 2002)
• To reduce post-analytic errors (Riebling 2005)
* J.M.Gras, M. Philippe. CCLM 2007;45(6):789-796.

1. Six Sigma Basics
• Belgian labs have also implemented Six Sigma for various
projects:
• KUL: DMAIC application to increase TAT for samples
sent from external laboratories
(Wouters et al, presented at Quality in the Spotlight Conference in March
2008)
• UCL: Lean and Six Sigma application to increase TAT in
an university corelab
(Fillée et al, presented at Siemens Academy in September 2010)

1. Six Sigma Basics
• Other applications concern the application of Six
Sigma as a tool for Internal QC management

2. Internal Quality Control Applications
Westgard started it all…
• Applications of Six Sigma in internal QC were
suggested by J.O.Westgard in 2005*
• Keys to understand these applications are:
1. It is possible to apply Six Sigma as a
quality indicator for laboratory medicine;
2. There is a link between Sigma levels and
power curves;
3. The Sigma level equation is based on the
process capability concept;
4. This application allows to adapt QC rules for
every test.
Clinique St-Luc Bouge* J.O. Westgard. Six Sigma Quality and Design.
Westgard QC Inc. 2005.

The Sigma level equation:
J.O. Westgard. Six Sigma Quality and Design.
Westgard QC Inc. 2005.
Sigma = [(TEa – biasmeas) /CV] *

Different Sigma levels mean different QC strategies
Free T4:
Sigma level= 2.81
Suggested QC Rule: 13s/22s/R4s/41S/8X (N=8)
TSH:
Sigma level= 6.30
Suggested QC Rule: 13s (N=2)

Meaning of the Sigma level equation:
1) In one equation, you find the quality objective, the bias and
the CV
2) It is a measure of the adequation between the quality
demand (TEa) and the test variability (CV)
3) Outstanding quality indicator (cfr ISO 15189)
4) It is recommanded to calculate the Sigma level at the
clinical decision treshold; in practice, we calculate one
sigma level per QC material

Sigma levels - practical questions – (1/4)
• Which TEa to use ?
• Various possibilities:
• Ricos norms (“desirable precision”)
• “Optimum” or “minimum” biological
variation
• CLIA
• Belgium: d from IPH
• RiliBäk
• Tonk’s rule
• ...
• In practice, try with Ricos norms first
Sigma = [(TEa – biasmeas) /CV]

Ricos TEas available on Westgard website
http://www.westgard.com/biodatabase1/html

• Which CV to use ?
• Validation period CV
• Cumulated CV (1 week, 1 month, 3 months,...)
• ...
• In practice, use a CV that is representative of the
lab’s real life (that includes maintenance and reagent
changes
• A two months cumulative CV is great

• Which Bias to use ?
• Bias as compared to reference method
• Peer-related bias
• Method related bias
• Currently, the peer-related bias is considered as the
most representative...and the easiest to obtain

Sigma levels - practical example
How to perform Six-Sigma based QC for one test ?
1) Select one test
2) Select a QC material
3) Define TEa
4) Evaluate method: CV, bias and Sigma calculation
5) Calculate OPSpecs and Power Function Graph
6) Calculate Ped and Pfr, choose Ped >90 % and Pfr <5%
7) Put it in practice
8) Evaluate the results

Several steps
1) Select one test
• By example
Luteinizing Hormone (LH)
2) Select one QC material
• By example
Bio-Rad LyphoCheck Immunoassay plus
3) Choose TEa
• Can we use biological variation ?

• Can we use biological variation based TEa ?
• Look at analytical CV (CVa) and compare to CVi
• Analytical CV=
• Level 1: 1,91 %
• Level 3: 1,4 %
• CVi (Ricos)= 14,5 %
• Comparison CVa/CVi: ratio CVa/CVi =
• Level 1: 0,131 (<0,25)
• Level 2: 0,096 (<0,25)
BV based TEa is applicable

4) Evaluate method:
a) CVa, bias
• CVa:
• Level 1: 1,91 %
• Level 3: 1,4 %
• Bias (comparison to peer group):
• Level 1: 0 %
• Level 3: 2,44 %

4) Method evaluation:
b) Sigma levels
• Level 1:
• BV desirable = (19,8-0)/1,91= 10,37
• BV optimum = 5,18
• Level 3:
• BV desirable = 12,40
• BV optimum = 5,33

5) Calculate OPSpecs and PowerFunction graphs
LH
Level 1

5) Calculate OPSpecs and PowerFunction graphs
LH
Level 3

6) Choose QC rule
7) Put that rule into practice
8) Evaluate results:
1. Follow evolution of Sigma results
2. Number of QC results rejected
3. Peer- group comparison
4. Complains from customers ?
5. External QC performance ?
13s (N=2)

Protocols in different labs
University lab
Regional lab

Examples of performance- university lab
Test CVi
(%)
CVa
(%)
Precision Bias
(%)
TEa (%)
Ricos
Sigma
Ricos
TEa (%)
tailored
Sigma
tailored
Rule
Albu 3.1 1.83 Min -0.06 3.9 2.1 7.37 3.99 L Multi
(n=4)
ALP 6.4 3.79 Des -3.83 11.7 2.08 11.7 2.08 Multi
(n=4)
ALT 24.3 3.56 Opt -1.01 32.1 8.73 20.2 5.39 S 13s
(n=2)
Amy 8.7 1.95 Opt -0.11 14.6 7.43 9.44 5.78 L 13.5s
(n=2)
AST 11.9 2.95 Opt -1.59 15.2 4.61 9.62 2.72 S Multi
(n=4)
UCL data, March 2008
Presented at QITS meeting, Antwerpen, March 2008

Examples of performance- regional lab
CSL Arlon data, 2008-2009
Presented at AACC 2009, Chicago, USA
Clinical Chemistry 2009; 55(S6):A32 (Abstract A-97).
Testing Control
material
Frequency TEa TEa type Sigma
level
QC rule applied
Lactate Multiqual 2 times/
day
15% <BV desirable 10 1-3s
Uric Acid Multiqual 1 time/ day 14.8 % BV desirable 12.1 1-4s
Albumin Multiqual 1 time/ day 7,5 % User defined 4.3 1-3s
Amylase Multiqual 1 time/ day 14.6 % BV desirable 11.8 1-4s
ASLO Multiqual 1 time/ day 15 % User defined 4.6 1-3s
Conj. Bilirubin Multiqual 1 time/ day 44.5 % BV desirable 22.3 1-4s
Total Bilirubin Multiqual 1 time/ day 31.1 % BV desirable 10.4 1-3s
Calcium Multiqual 2 times/
day
5 % User defined 5.9 1-3s/2-2s/R4s
Total Chol Multiqual 1 time/ day 9 % BV desirable 4.6 1-3s/2-2s
Chol- HDL Multiqual 1 time/ day 11.1% BV desirable 4.6 1-3s/2-2s

iQC applications summary:
1. Six Sigma can be applied as a quality indicator of
analytical performance
2. Sigma values will provide a summary of test
performance that is a reflexion of quality demand
and test variability
3. Different sigma values mean different QC rules
4. Application of Six Sigma to design QC procedures
has been performed in various lab types

- PLAN -
1) Six Sigma basics
2) Internal Quality Control applications- the local
perspective
3) External Quality control applications- the
national perspective
4) Tendencies and controversies in the use of Six
Sigma in clinical laboratories

3. External Quality Control Applications
Six Sigma based indicators for national QC programs
• Definition of 3 estimates of quality based on Sigma
metrics :
1) National Test Quality (NTQ),
where NTQ= TEa / CVgroup
2) National Method Quality (NMQ),
where NMQ= [TEa-Biasms] / CVms
3) Local Method Quality (LMQ),
where LMQ= TEa / CVmethodsubgroup
Clinique St-Luc BougeWestgard JO and Westgard SA. Am J Clin Pathol 2006; 125:343-354

Six Sigma based indicators for national QC programs
• Interest to develop such indicators in Belgium ?
• Use of IPH external quality assessment data
• Use of CLIA TEa, comparison with Ricos tables

Belgian indicator : Glucose (GM :14.54 mmol/L – 262 mg/dL)
TEa CLIA = 10 % TEa Ricos: 6.9 %
Using IPH data from 4th Trimester 2007
C/7930 CV (%) LMQ CLIA LMQ Ricos NTQ
1. Hexokinase (N=151) 2.7 3.7 Sigma 2.55 Sigma
2. Glucose deshydrogenase/
NAD (N=1)
/ / /
3. Glucose oxydase PAP (N=6) 2 5 Sigma 3.45 Sigma
4. Glucose oxydase O2 elect.
(N=18)
3.4 2.94 Sigma 2.02 Sigma
5. Reflectance photometry
(N =35)
2.3 4.34 Sigma 3 Sigma
TOTAL RESULTS (N=211) 2.7 CLIA: 3.7
RIcos: 2.5

Belgian indicator : Amylase TEa CLIA = 30 % TEa Ricos= 14.6 %
C/7930 CV (%) LMQ CLIA LMQ Ricos
1. Kinetic method-VIS photom
(N =1)
/ / /
2. Kinetic methods-VIS
photometry (chloro PNP
maltotrioside) (N =23)
3. Kinetic methods-UV
photometry (maltotetraose)
(N =16)
(amylopectin) (N =35)
5. Kin. meth-VIS photom. (PNP
maltoheptaosideethylidene)
(N =131)

Belgian indicator : Calcium (GM :2.29 mmol/L – 9.5 mg/dL)
TEa CLIA = 10.5 % TEa Ricos: 2.4 %
C/7930 CV (%) LMQ CLIA LMQ Ricos NTQ
1. VIS photometry (o-
cresolphtalein) (N=127)
(arsenazo III) (N =34)
3. VIS photometry (arsenazo
III) (N =28)
2.4 4.4 Sigma 1 Sigma
4. Indirect potentiometry
(N = 17)
TOTAL RESULTS (N=206) 2.4 CLIA:4.37
RIcos: 1

eQC applications summary:
1. Six Sigma can be applied as a quality indicator of
analytical performance on a national level
2. Methods giving close CVs have in fact a different
Sigma performance- differences in methods are
amplified by a Sigma level calculation
3. Use of Ricos TEas for national QC indicators need
conscious interpretation...

4. Tendencies and Controversies
Tendency 1:
Application of Sigma levels as ISO 15189 quality
indicators (1/2):
• Quality control procedures are the subject of a QSE
(Quality Systems Essential) in the ISO 15189 norm
• It is the QSE 5.6 (« Assuring Quality of Examination
Procedures »)

Tendency 1:
Application of Sigma levels as ISO 15189 quality
indicators (2/2):
• Some important points of QSE 5.6:
• QC results must be recorded
• QC design and analytical objectives must also be
recorded
• Comparison with peer group is useful and wished
• Many of these points are included in a Sigma value;
moreover, Sigma levels for each test are interesting to
monitor test performance over time

Tendency 2:
Groups of tests with similar Sigma performances
are monitored using similar QC rules (1/2):
• Many labs find similar groups of tests having similar
Sigma levels and that can be monitored by similar QC
rules
• These groups allow an intelligent adaptation of QC rules
and the establishment of feasible QC protocols

Tendency 2:
Groups of tests with similar Sigma performances are
monitored using similar QC rules (2/2):
Group 1
Sigma > 6
13s
QC once a day
Group 2
Sigma > 4.5
13s
QC twice a day
Group 3
Sigma > 3
13s/22s/R4s/41s/10x
QC once a day
Group 4
Sigma < 3
13s/22s/R4s/41s/10x
QC twice a day
Or
Combination
with other QC
strategies
QC rules may vary according labs’ QC philosophy

Tendency 3:
Combination of simple QC rules with less known rules:
• 7T QC rule: reject a run when 7 results are going the
same way (up or down): allows to easily detect a drift
when using “large” general QC rules such as 13s or 15s
• Application of classical rules accross two QC levels
(INTER-level application >< Inra-level application: may
be useful to react faster and take preemptive corrective
actions

Controversy 1:
Biological Variation based TEas are too stringent and not
applicable (1/2):
• BV based TEas may look too stringent at first glance
• However, it is possible to design a routine QC protocol
based on Six Sigma and using:
• BV based TEa (desirable precision)
• A two months cumulative CV
• Peer related bias

Controversy 1:
Biological Variation based TEas are too stringent and not
applicable (2/2):
TEa used for Clinical Chemistry tests
72%
5% 5%
18%
Biological Variation
desirable precision
User defined, more
stringent than BV
desirable precision
CLIA
User defined
TEa used for Immunoassays
82%
6%
12%
desirable precision
User defined
Not Applicable (no TEa
found)
29%
27%
12%
20%
8%
4%
desirable precision
User defined, more
stringent than BV
desirable precision
CLIA
More stringent than CLIA
User defined
Not Applicable
TEa used on the back up automate
CSL Arlon data, 2008-2009
Presented at AACC 2009, Chicago, USA
Clinical Chemistry 2009; 55(S6):A32 (Abstract A-97).

Controversy 2:
There are practical problems encountered with the
individuation of QC rules
• Adaptation of QC rules to every test may result in
complex QC protocols that are not applicable in practice
• The solution may be to divide tests in various groups
according to their performance on the Sigma Scale
• The aim is to create a QC protocol that is reflective of
each test Biological Variation characteristics and
analytical performance but that is feasible in routine

Controversy 3:
There are problems encountered with rules
oversimplification (1/2):
• For some tests with excellent Sigma performance, “loose”
QC rules (15s, by example) are sometimes proposed by
QC softwares
• These rules are clearly insufficient to detect small changes
in QC results

Controversy 3:
There are problems encountered with rules
oversimplification (2/2):
• Solutions:
1. Combination of simple rules with innovative rules
2. Do not use QC rules that are too loose
3. Combine classical statistical QC with other
concepts, by example, patient mean monitoring

Controversy 4:
There are unanswered questions with Six Sigma QC:
• Implication of the different types of BV based TEa
in a Six Sigma QC (optimum-desirable-minimum) ?
• How to perform QC for tests that are run on
different analysers ?
• What is the return on investment on such a QC
strategy ?
• Is statistical QC enough ?

Challenges when creating a Six Sigma QC protocol
• Review QC procedures for every test (>100 in a corelab)
• Create an advanced QC protocol that is feasible
• Gain acceptance by MLTs
• Evaluate the ROI
• Combine with other QC approaches (using patient data,
combination with risk management)

Conclusions
• Six Sigma quality management initiatives have been
succesfully launched in laboratories
• QC based on Six Sigma and biological variation based
analytical objectives is feasible
• Sigma levels for lab tests are excellent quality indicators
• Six Sigma based QC is certainly useful in the context of
ISO 15189 accreditation
• Sigma indicators could add information to a national QC
program
• Future deployment of Six Sigma based QC can answer
important questions, notably regarding ROI

Many thanks for your attention !
jeremie.gras@gmail.com

J.Gras Six SigmaTSIC 2010

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (16)

Semelhante a J.Gras Six SigmaTSIC 2010

Semelhante a J.Gras Six SigmaTSIC 2010 (20)

J.Gras Six SigmaTSIC 2010