1. Evaluation of
Molecular Typing Techniques
Thomas Weniger
Department of Periodontology
University of Münster, Germany
tweniger@uni-muenster.de
2. Typing:
Phenotypic and/or genetic
analysis of bacterial isolates,
below the species/subspecies
level, performed in order to
generate strain/clone-specific
fingerprints […]
van Belkum et al., (2007). CMI, 13:1
4. Questions Suitable methods
Discriminatory
power
Time
span
Outbreak investigations
Short-term/local surveillance
Control of hygiene measures
PFGE, RFLP,
AFLP, RA-PCR,
VNTR, SLST,
micro-array
high
weeks -
month
Long-term/global
epidemiological studies
Population genetics
Analysis of population-based
interventions, e.g. vaccination
(MLEE), MLST,
micro-array,
in part SLST
low years
6-AM 4
Typing questions & suitable methods
14. Typing system concordance
- Rand‘s index
- adjusted Rand‘s index
- Wallace coefficients
Carrico et al. (2006) JCM 44: 2524
PFGE versus MLST MLST
same different
PFGE
same a b
different c d
15. Rand (1971) J. Am. Stat. Assoc. 66
Hubert & Arabie (1985) J. Classification 2
21. Example DI
Method No. of types
No. of most
frequent type
Index of diversity
(95% CI)
PFGE 19 8 0.944 (0.909-0.979)
MLST ST 14 12 0.886 (0.825-0.948)
flaA 13 6 0.920 (0.895-0.944)
flaB 12 8 0.902 (0.871-0.934)
Typing: Analysis of bacterial isolates below species level, performed to generate strain-specific fingerprints.
To put it simply, typing applies distinct labels to bacterial isolates.
Fingerprint: A specific pattern displayed by an isolate on application of one or more typing methods.
see
van Belkum et al., (2007). CMI, 13:1
Example of genotyping methods for Campylobacter jejuni:
-Pulsed field gel electrophoresis (PFGE)
-Multi Locus Sequence Typing (MLST)
-flaA typing (Single Locus Sequence Typing, SLST)
How can the performance of different typing method be measured?
Note that different investigations may have different requirements, therefore there is no ideal, universally applicable bacterial typing method.
An example slide that compares different typing methods.
Author of slide: Alexander Mellmann
Evaluation and validation can be done by various criteria.
These can be divided into performance and convenience criteria.
Typability: a method’s ability to assign a type to all isolates tested by it.
- Typability should be high (100%)
- Can be lower for phenotyping methods, e.g. serotyping (schemes may not cover all genotypes)
An isolate’s fingerprint should
- not change rapidly
- based on the epidemiological context
- results should remain stable during laboratory storage and subculture
- multiple subcultures of same isolate > store under same conditions for different periods > same results?
- stable during study period
- depending on species
- not to be confused with reproducibility
- The ability of a typing method to assign
- the same type to an isolate tested on independent occasions,
- separated in time and/or
- Place
- Reproducibility may be influenced by
- the protocol used
- stringency of its application.
- analysis and interpretation of results.
- Reproducibility has intra-laboratory and inter-laboratory dimensions.
- should be high (100%)
-> Use standardized protocols and adequate personnel training to ensure a reliable method
- a method’s ability to assign a different type to two unrelated strains sampled randomly from the population of a given species.
- should be calculated using a test population that includes epidemiologically unrelated strains
Discriminatory power can be expressed as a probability using Simpson‘s index of diversity
- Simpson’s index of diversity should be 95% to 100% (according to van Belkum)
- Do critical assessment of the confidence interval
see
van Belkum et al., (2007). CMI, 13:1
Example
DI for typing method 1 is 0.933 [0.805 – 1.0]
compared to typing method 2: DI: 0.733 [0.53 – 0.936]
=> method 1 is more discriminatory
How well does the typing method fit into the epidemiological context?
The results of a typing method should reflect and agree with the available epidemiological information
For example, epidemiologically related isolates derived from single-strain outbreaks should be assigned to identical or related types.
A good typing method should assess a marker that
- should be testable in every isolate, i.e., it should provide universal typeability of all isolates.
- remains stable during the study period
- should be reproducible, independently of the operator, place and time
- usefully discriminates among isolates
- This discrimination should be concordant with the epidemiological picture
Cost:
- initial equipment (e.g. sequencer)
- costs of servicing
- cost of consumable reagents
- staff costs (work time, training time)
Rapidity:
- time required to get from the bacterial isolates to the final typing results
- faster if method can be applied directly on clinical material
Ease of use:
- technical simplicity
- suitability for processing large numbers of isolates
- ease of interpreting the results
- amenability to computerised analysis
- ability to incorporate typing results in electronic databases
Compare the result of two typing systems.
All these coefficients are calculated based on how pairs of isolates are grouped in both compared typing methods
see
Carrico et al. (2006) JCM 44: 2524
Rand’s Coefficient
- Rands’s coefficient: proportion of agreement for both matches (a) and Mismatches
- Rand’s coefficient is not zero for random data sets
-> Adjusted rand addresses this issue and corrects for change agreement.
see
Rand (1971) J. Am. Stat. Assoc. 66
Hubert & Arabie (1985) J. Classification 2
Wallace coefficients.
Wallace’s coefficients W1 and W2
- Assume that a typing method is the standard.
- Probability that a pair of points which are in the same cluster under P are also in the same cluster under P’ (and vice versa)
- Provide an estimate of how much new information is obtained from another typing method.
- High values indicate that partitions defined by a given method could have been predicted from the results of another method, suggesting that the use of both methodologies is redundant:
see
Wallace (1983) J. Am. Stat. Assoc. 78
Adjusted Rand‘s is low, and W1 is high-> method 2 adds no new information
Be careful with missing data!
Do NOT use “no category”, “NT” or negative values.
Use empty space instead.
Otherwise, the software might think that “no category” or “NT” is an own category.
Example from Mellmann et al. 2004
see
Mellmann et al. (2004). JCM 42: 4840