this lecture describes the methods used and recommended by CLSI to validate both the qualitative and semi-Quantitative laboratory test methods

1. Validation of Qualitative Lab
Test Methods
By: Dr Mostafa Mahmoud MD,
Consultant Microbiologist
Labs & Blood Banks Admin, Riyadh, MOH
Associate Professor of Medical Microbiology & Immunology
Faculty of Medicine – Ain Shams University

2. Procedures for validation
• Each method validation/ verification study is a collection of
experiments to assess performance and error in order to judge a
method’s suitability for use in the laboratory.
• Acceptability Criteria
• the laboratory must establish acceptance criteria.
• Parameters for accuracy, precision, sensitivity and specificity should
include a confidence level of at least 90%, or meet the claims of the
manufacturer.

3. Qualitative Methods:
• Includes semi quantitative testing that use cut offs such as hepatitis
testing and some molecular testing.
• No values/concentrations are included in the patient report.
• Test results are reported as positive/negative, normal/
borderline/abnormal, reactive/nonreactive, detected /not detected.
•1. FDA cleared or approved methods (CLIA Standards).
• Parameters to be validated:
1. Accuracy (correlation or comparison)
2. Precision (Replication)
3. Reportable range
4. Reference range (Normal values)

4. 1- Accuracy (correlation or comparison)
• Demonstrates how close to the “true” value the new method can
achieve.
The materials (Samples) used:
1- Calibrators/controls,
2- Reference material,
3- Proficiency testing material with known values,
4- Samples tested by another lab using the same or similar method,
5- or by comparing results to an established comparative method.

5. Method & Number of samples:
• Most sources recommend comparing at least 40 patient specimens.
CLIA current guidance suggests a minimum of 20 samples.
• A method comparison must be used for 5 days and for another 5
days if failed.
• Document the results of the new method comparing the known
values from the reference sources, another certified laboratory’s
results or with results from the current method.

6. Calculations:
• Calculate the percent of positive, negative and total accuracy by
dividing observed results over known results multiplied by 100.
• Example: New method = 19 positives, 20 negatives. Current
method or reference material with known values = 20 positive, 20
negatives
• Percent positive accuracy 19/20 X 100 = 95%
• Percent negative accuracy 20/20 X 100 = 100%
• Total accuracy 39/40 X 100 = 98%

8. 2- Precision (Reproducibility ):
• Can the new method duplicate the same results?
•Materials used:
1- patient samples are the first choice followed by
2- control material and reference solutions.
Method & Number of samples:
- 2 negative samples and 2 positive samples run in triplicate for 5
days will provide data for within-run and between-run
components to estimate precision.
•

9. Calculation:
• Calculate the percent within-run (intra), between-run (inter) and
total precision by dividing observed results over known results
multiplied by 100.
• See next tables.

10. ID Day 1 Day 2 Day 3
Pos sample Pos Pos Pos Pos Pos Pos Pos Pos Pos
Pos sample Pos Pos Pos Neg Pos Pos Pos Pos Pos
Neg sample Neg Neg Neg Neg Neg Neg Neg Neg Neg
Neg sample Neg Neg Neg Neg Neg Neg Neg Neg Neg
Within run % 12/12/x 100 = 100% 11/12 x 100 = 92% 12/12/x 100 =
100%

11. ID Day 4 Day 5 Between run %
Pos sample Pos Pos Pos Pos Pos Pos 15/15x100 = 100%
Pos sample Pos Pos Pos Neg Pos Pos 14/15x100 =93%
Neg sample Neg Neg Neg Neg Neg Pos 14/15x100 =93%
Neg sample Neg Neg Neg Neg Neg Neg 15/15x100 = 100%
Within run % 12/12/x 100 = 100% 11/12 x 100 = 92%
Total Precision: 58/60 x 100 = 96.7%

12. 3- Reportable Range:
• The highest and lowest test values that can be analyzed while
maintaining accuracy.
• For tests without high or low values, define method criteria for a
positive result.
Method & Number of samples:
• To verify reportable range, test at least 3-5 low and high positive
samples once.
• These samples can be combined with the accuracy/precision
experiments.
• Include both weak and strong positive samples.

13. • For methods depending on a cut-off value to determine positive
results, testing positive specimens near the cut-off can serve as
the cut-off validation.
•CAP Requirement:
• For qualitative tests that use a cut-off value to distinguish
positive from negative, the cut-off value is established initially,
and verified every 6 months thereafter.
• N.B. This requirement does not apply to FDA-cleared/approved in
vitro diagnostic assays that report the qualitative result based on a
predefined cut-off value.
• Applied to laboratory established threshold (cut-off value)

14. 4- Reference Range (Normal Values):
• Provided by the manufacturer and verified by running known normal
patients.
• Manufacturer’s ranges or even published reference ranges in
textbooks are used if the patients are similar.
•Method & Number of samples:
• The Reference Range can be verified by testing 20 known normal
samples; if no more than 2 results fall outside the
manufacturer/published range then that reference range can be
considered to be verified.

15. • If the laboratory cannot reference the normal values, then the
reference range will need to be established. This involves a
selection of at least 120 reference samples for each group or
subgroup that needs to be characterized.

16. 5- Sensitivity & Specificity:
• CLIA does not require that these parameters to be verified.

17. Validation Summary for FDA-approved qualitative tests:
• Clearly state the purpose of the verification, what
platform/method and the number of samples for each experiment.
Any discrepant results should be investigated and explained in the
Summary. Test results that show sample problems such as
contamination and degradation should not be used in the
assessment but still listed with an explanation.
• The Summary should also contain a Conclusion stating weather the
study met the acceptance criteria or not and its suitability for us in
the laboratory.
• Additional samples to be added if results are outside acceptance.
• If the results failed then test not to be implemented.

18. 2. Non-FDA Cleared tests
• Qualitative methods developed in-House, non-FDA cleared
methods and FDA-cleared methods modified by the laboratory.
• - For Qualitative methods follow the instructions above for
Accuracy, Precision, Reportable Range and
Reference Range.
•Qualitative Sensitivity is not done by analytic sensitivity
but by diagnostic sensitivity.
• Diagnostic sensitivity: The percent of subjects (patients) with the
target condition whose test values are positive.

19. •Method &number of samples:
• dividing the number of true positives by the sum of the number of
true positives plus the number of false negatives and multiplying by
100.
Diagnostic Sensitivity: = [TP ÷(TP + FN)] x 100
- If a person has a disease, how often will the test be positive (true
positive rate).
- The higher the sensitivity the lower the False Negative and vice
versa.

20. •Qualitative Specificity
• Qualitative sensitivity not assessed by Analytical Specificity but
also by the Diagnostic Specificity.
• Diagnostic Specificity: the percent of subjects (Persons) without
the target condition whose test values are negative.

21. •Method & number of samples:
• Calculate by dividing the number of true negatives by the sum of
the number of true negatives plus the number of false positives
and multiplying by 100.
Diagnostic Specificity: [TN ÷(TN + FP)] x 100
-If a person does not have the disease, how often will the test be
negative ( true negative rate).
- The higher the specificity the lower the False Positive and vice
versa.

22. Interference:
• The laboratory must be aware of common interfering substances by
referencing studies performed elsewhere (manufacturer or literature)
or by performing studies.
• Common blood interferences are; hemolysis, bilirubin, lipemia,
preservatives and anticoagulants used in specimen collection.

23. CAP Accreditation requirements for validating
laboratory tests
FDA approved/cleared LDTs & modified FDA tests
Accuracy & Precision
(COM.40300, ph II)
Verify manufacturer’s results Establish (= validation)
Analytic sensitivity (LOD)
(COM.40400, ph II)
Verify: manufacturer or
literature documentation OK
Establish
Analytic specificity
(interferences)
(COM.40500, ph II)
Reference literature or
manufacturer documentation
Establish; studies by
manufacturer or in literature
OK
Reportable range (AMR)
(COM.40600, ph II)
Verify* Establish*
Reference range
(COM.50000, ph II)
Verify ** Establish**

24. Test method validation CBAHI standard
LB.10 The laboratory develops a process for test method validation.
LB.10.1 The laboratory implements policies and procedures on
test method validation including:
LB.10.1.1 Verification of accuracy/precision.
LB.10.1.2 Verification of sensitivity (lower detection limit).
LB.10.1.3 Verification of carryover acceptability.
LB.10.1.4 Verification of the Analytic Measurement (AMR)
LB.10.1.5 Approval of the method for clinical use (CRR??).

25. Test Method Validation
Evidence of compliance to (LB 10) standard:
1- Document Reviews:
- Policies, process and procedures on method validation.
2- Documented Evidences
- Records surveyor-selected method confirms compliance with
policies and procedures.
3- Staff Interview
- Senior personnel are knowledgeable about the concept of
method validation.

26. • LB.13 The laboratory has a system for instruments/methods
correlation.
LB.13.1 When the laboratory uses more than one method and/or
instruments to test for a given analyte, the laboratory develops
and implements policies and procedures on correlation to ensure
the following:
LB.13.1.1 The correlation studies are conducted every six months.
LB.13.1.2 There is clear description of the correlation study.
LB.13.1.3 There are clearly defined acceptance criteria.
LB.13.1.4 There is a process for review and approval of the
correlation results.

27. • Evidence of Compliance (LB.13):
• Document Review
- Policies, process and procedures on methods/instrument
correlation.
• Documented Evidences
- Records surveyor-selected methods/instruments confirms
compliance with policies and procedures.
• Staff Interview
- Senior personnel are knowledgeable about the purpose of
methods/instrument correlation.

28. • LB.14 The laboratory has a system for controlling the quality of test
methods.
LB.14.1 The laboratory implements policies and procedures on quality
control of test methods to satisfy the following:
LB.14.1.1 Assignment of performance and review responsibility (control
specimens are handled and tested in the same manner and by the same
laboratory personnel testing patient samples).
LB.14.1.2 Number and frequency of running controls.
LB.14.1.3 Tolerance limits of controls results.
LB.14.1.4 Corrective action to be taken in the event of unacceptable results.
LB.14.2 The laboratory quality control system conforms to the
manufacturer's instructions.

29. •Evidence of compliance to LB 14
• Document review:
- Policies, process and procedures on controlling the quality of test
methods.
- Policies and procedures on controlling the quality of test methods
confirm to the manufacturer instructions.
• Documented Evidences:
- Records surveyor-selected methods confirms compliance with
policies and procedures.
• Staff Interview:
- Laboratory personnel understanding of the purpose of controlling the
quality of test methods.