INTER-LAB COMPARISON /
PROFICIENCY TESTING AND
ESTIMATION OF SCORES

Measurements by labs
 Trillions of measurements daily worldwide
 Aspects of measurement
 Quality
 Repeatability and reproducibility
 Consistency
 Customer’s specification
 And many more

Measurement Results
 Lab A measurement
results
 x
 x + µ
 x - µ …..
 Results are consistent,
repeatable & reproducible
 Lab B measurement results
 y
 y + ∂
 y - ∂ …..
 Results are consistent,
repeatable & reproducible
 Although the results of the same sample, the results of Lab A or Lab B
may or may not be consistent with each other, even if measured by same
operator in the lab.
 The results of Lab A or Lab B may or may not be consistent with each
other, if measured by different operators in the lab.
 More so, the results of Lab A or Lab B may or may not be consistent with
each other when compared with each other.

Why ILC/ PT ?
© Anuj Bhatnagar
4
 First: To ensure that a single operator / analyst within a
laboratory is able to consistently reproduce the same
result on the same sample.
 Second: The result produced by this operator/ analyst
should reflect the result that would have come from any
other analyst in the laboratory.
 Third: Any results from the laboratory as a whole should
reflect the results that are agreed upon by many other
laboratories.

Inter-Lab Comparison /Proficiency
Testing
 It is one of the techniques by which the data produced by
a lab/ method/ software/ procedure is validated.
 It is means of quality control for objectively assessing the
quality control procedures of a lab.
 Objective means of assessing the reliability of data they
are producing.
 It is method to check a lab’s testing / calibration
performance.
 Performance within its own lab and with other labs.
 Provide additional confidence to the lab’s customers.
 It is pre Accreditation activity.

Users of ILC/ PT
 Participant laboratories, Accreditation bodies,
 Regulatory authorities,
 Clients of the labs
 Self evaluation of labs
 Values to CRMs
 ILAC/ APLAC/ EA MRA partners
 Standards Bodies

PT provider/
Coordinator
Accreditation
Bodies
Professional
Bodies
ILAC/ APLAC
MRA Partners
Laboratories
Ref. Material
Producers
Indirect
Customers
(Regulators)
Direct
Customers
(Users)
Users of ILC/ PT

ILAC Policy (ILAC P-9)
“Accreditation bodies seeking to sign or seeking to
maintain the status as signatory to the ILAC MRA need to
demonstrate the technical competence of their accredited
calibration and testing laboratories. One of the elements
by which accredited calibration and testing laboratories
demonstrate technical competence is by satisfactory
participation in PT activities where such activities are
available”.
 Recommended minimum appropriate PT activities per
laboratory is:- one activity prior to gaining accreditation
- one activity relating to each major sub
discipline of a labs scope of accreditation with in four
years.

Inter-Lab Comparisons
Widely used for a number of purposes
Typical purposes for inter-laboratory comparisons include:
1. Evaluation of the performance of laboratories for specific tests or
measurements and monitoring laboratories' continuing performance;
2. Identification of problems in laboratories and initiation of actions for
improvement which, for example, may be related to inadequate test or
measurement procedures, effectiveness of staff training and supervision,
or calibration of equipment;
3. Establishment of the effectiveness and comparability of test or
measurement methods;
4. Provision of additional confidence to laboratory customers;

5. Identification of inter-laboratory differences;
6. Education of participating laboratories based on the outcomes of
such comparisons;
7. Validation of uncertainty claims;
8. Evaluation of the performance characteristics of a method – often
described as collaborative trials;
9. Assignment of values to reference materials and assessment of their
suitability for use in specific test or measurement procedures; and
10. Support for statements of the equivalence of measurements of
National Metrology Institutes through “key comparisons” and supplementary
comparisons conducted on behalf of the International Bureau of Weights and
Measurement (BIPM) and associated regional metrology organizations.
Inter-Lab Comparisons

Proficiency Testing
Proficiency testing involves the use of inter-laboratory comparisons for
the determination of performance of the laboratory and that of its testing
personnel/ analysts/ operators, as listed in 1) to 7) above.
Proficiency testing does not usually address 8), 9) and 10) because
laboratory competence is assumed in these applications, but these
applications can be used to provide independent demonstrations of
laboratory competence.
However, the requirements of International Standard ISO 17043 can
be applied to many of the technical planning and operational activities for
8), 9) and 10).

Proficiency Testing
© Anuj Bhatnagar
12
 The precise way is not prescribed in ISO/IEC 17025,
although it refers to the use of proficiency testing at
couple of places.
 The process is clearly defined/ explained in detail in
some other international standards
 International Standards used for this purpose:
 ISO 17043
 Guides made by various Accreditation Bodies available
on internet

Clause 5.4.5 of ISO/IEC 17025
© Anuj Bhatnagar
13
(Cl 5.4.5.2): Lab Should Validate….
 Non-standard methods,
 Lab designed/developed methods,
 Standard methods used outside of their intended scope,
 Amplification and modification of standard methods,
 Procedure for sampling, handling and transportation.

© Anuj Bhatnagar
14
By use of:
 Calibration using Ref standards or Ref material
 Comparison of results achieved with other methods
 Inter-laboratory comparison or Proficiency Testing
 Systematic assessment of factors influencing result
 Assessment of the uncertainty of results based on
scientific understanding of the theoretical principles and
practical experience.

Clause 5.9 of ISO/IEC 17025
© Anuj Bhatnagar
15
 Assuring the Quality of Test and Calibration
results: …. This monitoring shall be planned and
may include, but not limited to:
(b) participation in interlaboratory comparison or
proficiency-testing programmes

ISO/IEC 17043
© Anuj Bhatnagar
16
ISO/IEC 17043 is the new international standard to
determine the competence of organizations that provide
proficiency testing.
ISO/IEC 17043 is published in 2010
Replaces ISO/IEC Guide 43-1 and 43-2, but preserves and
updates the principles for the operation of proficiency
testing described in ISO/IEC Guides.

© Anuj Bhatnagar
17
 …directed to the providers of proficiency testing
schemes who wish, on a voluntary basis, to
demonstrate their competence, for the purpose
of accreditation or other recognition…
 …. Only to the use of inter-laboratory
comparisons for the purpose of proficiency
testing…

Till date, we had
© Anuj Bhatnagar
18
 ISO/IEC Guide 43-1
Proficiency Testing by Inter-laboratory Comparisons.
Part 1: Development and operation of proficiency
testing schemes
International Organization for Standardization
Geneva, First Edition 1997
 ILAC- G13
Guidelines for the requirements for the Competence
of Providers of Proficiency Testing Schemes
ILAC Technical Accreditation Issues Committee, 2007
(latest revised
issue)

Actual work on standardization for accreditation of
proficiency test providers is mainly focused on
these two documents
© Anuj Bhatnagar
19
ISO /IEC
Guide
43-1
ILAC –
G13
ISO/IEC
17043

ISO/IEC 17043
© Anuj Bhatnagar
20
 Content Structure:
1. Scope
2. References
3. Terms and Conditions
4. Technical requirements
5. Management Requirements
6. Appendix
Unlike ISO/IEC 17025, technical requirements get
priority over management requirements in ISO/IEC
17043.

ISO/IEC 17043
© Anuj Bhatnagar
21
 New Topics and additions:
4. Technical Requirements
4.4.5 Assigned Values
4.7 Data analysis and interpretation of
proficiency
testing scheme results
Annex A: Types of proficiency testing schemes
Annex C: Selection and use of proficiency testing
schemes

Other documents useful in PT
© Anuj Bhatnagar
22
 Eurachem Guide : 2010 (for PTs in food sector)
Selection, use and interpretation of proficiency testing
(PT)
by laboratories
 Eurachem Guide (2011) on Selection, Use and
Interpretation of Proficiency Testing (PT) Schemes For
methods for evaluation of proficiency testing
 ISO 13528
 Statistical methods for use in proficiency testing by
inter-laboratory comparisons
 ISO 5725: 1 to 6
 Accuracy (trueness and precision) of measurement
methods

Terms and Definitions
© Anuj Bhatnagar
23
 Assigned value: value attributed to a particular
quantity and accepted, sometimes by
convention, as having and uncertainty
appropriate for a given purpose
 Accepted reference value: a value that serves
as an agreed upon reference for comparison and
which is derived as (i) a theoretical or
established value, based on scientific principles,
(ii) an assigned value, based on experimental
work of some national or international
organization, and (iii) a consensus value, based
on collaborative experimental work under the
auspices of a scientific or engineering group.

Accuracy and precision
The dictionary definition of both ‘accuracy’ and ‘precision’ are
roughly the same, indicating that these words may be used
synonymously. However in ‘Analytical Science’ they have two
separate meanings, the difference between them is best
illustrated by using target diagrams
Poor precision
poor accuracy
Good precision
poor accuracy
Good mean accuracy
poor precision
Good accuracy
good precision

Accuracy and precision (2)
You saw from the previous slide, a set of results can be
either accurate and/or precise or can be neither accurate
nor precise. Thus accuracy may be defined as:
The closeness of the mean value from a replicate set of results to the
true or accepted value
Precision may be defined as:
The spread of results from a replicate set of measurements

© Anuj Bhatnagar
26
 Reference laboratory: laboratory that provides
reference values on a test item.
 Reference material (RM): Material or substance, one or
more of whose property values are sufficiently
homogeneous and well established to be used for the
calibration of an apparatus, the assessment of
measurement method, or for assigning values to the
materials.

© Anuj Bhatnagar
27
 Certified reference material: reference material,
accompanied by a certificate, one or more of whose
property values are certified by a procedure which
establishes its traceability to an accurate realization of
the unit in which the property values are expressed, and
for which each certified value is accompanied by an
uncertainty at a stated level of confidence.
 Laboratory Bias: The difference between the population
mean of the test result and an reference value.

© Anuj Bhatnagar
28
 Inter-laboratory comparisons: organizations of
performance and evaluation of tests on the same or
similar test items by two or more laboratories in
accordance with predetermined conditions.
 Coordinator: organization (or person) with responsibility
for coordinating all of the activities involved in the
operation of a proficiency testing scheme.
 Customer: organization or individual for which a
proficiency scheme is provided through a contractual
arrangement.

© Anuj Bhatnagar
29
 Participant: laboratory, organization or individual that
receives proficiency test items and submits results for review
by the proficiency test provider
 Proficiency testing: evaluation of participant’s performance
against pre established criteria by means of interlaboratory
comparisons.
 Proficiency test item: sample, product, artefact, reference
material, piece of equipment, measurement standard, data
set or other information used for proficiency testing

© Anuj Bhatnagar
30
 Proficiency testing provider: organization which takes
responsibility for all tasks in the development and
operation of proficiency testing scheme.
 Proficiency testing round: single complete sequence of
distribution of proficiency test items, and the evaluation
and reporting of results to the participants
 Proficiency testing scheme: proficiency testing
designed and operated in one or more rounds of a
specified area of testing, measurement, calibration or
inspection

© Anuj Bhatnagar
31
 Outlier: member of a set of values which is inconsistent
with the other members of that set.
 Extreme results: outliers and other values which are
grossly inconsistent with other members of the data set.
 Robust statistical techniques: techniques to minimize
the influence that extreme results can have on estimates
of the mean and standard deviation.
 Variance: A measure of dispersion, which is the sum of
the squared deviations of observations from their
average divided by one less that the number of
observations.

© Anuj Bhatnagar
32
 Standard deviations: The positive square root of the
variance.
 Subcontractor: organization or individual engaged by the
proficiency test provider to perform activities (on behalf of the
provider) that effect the quality of proficiency testing scheme.
 Uncertainty of measurement: parameter associated with
the result of a measurement, that characterizes the dispersion
of the values that could reasonably be attributed to the
measurand.

© Anuj Bhatnagar
33
 Random error: result of a measurement minus the mean
that would result from an infinite number measurements of
the same measurand carried out under repeatability
conditions.
Random error is equal to error minus systematic error.
It is possible to determine only an estimate of random
error as only a finite number of measurements can be
made.
 Systematic error: Mean that would result from an infinite
number of measurements of the same measurand carried
out under repeatability conditions minus a true value of the
measurand.
Systematic error is equal to error minus random error.
Systematic error and its causes cannot be completely
known.

© Anuj Bhatnagar
34
 Repeatability: closeness of agreement between the results
of successive measurements of the same measurand
carried out under repeatability conditions.
The repeatability conditions are:
- the same measurement procedure
- the same observer
- the same measuring instrument, used under same
conditions
- the same location
- repetition over a short period of time.

© Anuj Bhatnagar
35
 Reproducibility: Closeness of agreement between the
results of the measurements of the same measurand
carried out under changed conditions of measurement.
A valid statement of reproducibility requires specification
of the conditions changed. A changed condition include:
- principle of measurement
- method of measurement
- observer
- measuring instrument
- reference standard
- location
- conditions of use
- time

 Metrological Traceability
The property of a measurement whereby the result
can be related to a reference through a documented
unbroken chain of calibrations, each contributing to
the measurement uncertainty.
© Anuj Bhatnagar
36

Case 1
One of the labs (Accredited lab) is testing PVC insulated
cables as per IS 694:1990. In the Insulation Resistance test
(IR), a sample of 3 m is kept immersed in water and IR is
calculated. The lab had been doing this test for quite some
time.
When the lab participated in the PT program for cable test,
its score was unsatisfactory in this particular test. On
examining it was observed that the lab was taking only 3m
and immersing it in water with both ends (app.10cm) kept
out of water. Effective immersion is then about 2.8 m and
not 3 m as required.
PT test in this case helped the lab in finding out this
discrepancy.

Case 2
In the PT test conducted by an Accredited Lab for testing of
electric Irons as per IS 366, one of the participating lab
reported very high values for the temperature rise of the
knob. This lab had an unsatisfactory score and on
examination, it was observed that the temperature rise of
the knob was being taken by this lab after removal of the
knob, which is actually not permitted.
The lab was informed about this and later it modified its
procedures for conducting this test.

Case 3
One of the oil company in the public sector was using a
method of testing Lube Oil (determining impurities of Lube
Oil) which was actually developed by Shell. The oil
company wished to use this method after making some
modifications in the original method developed by Shell,
owing to the different ambient conditions prevailing in
India.
The company made the modifications in the original
procedure and used inter-lab comparison program
involving 20 labs for validating this procedure.
The method / procedure was validated using PT
techniques and now the company is using this procedure
for their purpose. It has also obtained NABL accreditation
for their labs in India and this method.

Case 4
One of the accredited laboratory participated in PT
organized by APLAC and received a sample of small
transformed. The lab tested the transformer for certain
tests. When the report was received, it came out that the
lab’s result was unsatisfactory (i.e. more than 3). It was
required to take corrective action and inform the country’s
Accreditation Body. It did so but the corrective action was
not found to be very effective by APLAC.
This resulting in the accreditation of the lab withdrawn by
the Accreditation body.