Diagnostic accuracy of MALDI-TOF mass spectrometry for the direct identification of clinical pathogens from urine Journal Club (Systematic Review & Meta Analysis) Clinical Microbiology Fellowship

1. Systematic Review & Meta analysis:
Diagnostic accuracy of MALDI-TOF mass
spectrometry for the direct identification of
clinical pathogens from urine.
Clinical Microbiology Resident.
King Fahd Hospital of the University.
Teaching Assistant, Department of Microbiology, Imam Abdulrahman Bin
Faisal University, Dammam, Saudi Arabia.
Abdullatif Sami Al Rashed

2. Why this publication?

3. Outlines
Introduction and aim
Material and Methods
Results
Conclusion
Critical Appraisal

4. Introduction
Urinary tract infections (UTIs) are among the most common clinical infectious
diseases and are also a major cause of hospital-acquired infections
Consequently, urine cultures are one of the most frequently ordered tests in
clinical microbiology laboratories
The identification of microorganisms in clinical microbiology laboratories is
currently based on Gram staining, culture and growth characteristics, and
biochemical patterns.
Given that culture requires 18 to 24 h for pathogen growth and an additional 18
to 48 h for identification and antimicrobial susceptibility testing results, rapid
alternatives are needed to streamline therapy.
Flores-Mireles AL et al. 2015

5. Introduction
• Matrix-assisted laser desorption ionization time-of-flight (MALDI-
TOF) mass spectrometry (MS) is a reliable method for the
identification of microorganisms from culture colonies.
• The use of MALDI-TOF MS for microorganism identification from
direct clinical samples has been rare until recently, mainly because
broad microorganism protein databases have become available.
• Various institutions started to apply direct microorganism
identification, which aids in providing timely antibiotic therapy and
decreasing unnecessary antibiotic use.
• In light of its promise, MALDI-TOF MS technology was described
as “a revolution in clinical microbiology”
Jean-Philippe L et al. 2013

6. Introduction
• Currently, two main commercial systems are popular for use
worldwide:
– The MALDI MS Biotyper (Bruker Daltonics, Germany)
– The Vitek MS (bioMérieux, France)
Jean-Philippe L et al. 2013

8. Aim
Therefore, the present work aimed to analyze and compare the
performance of MALDI-TOF MS with the common methods for
the diagnosis of pathogens from urine samples by performing a
meta-analysis to improve the reliability of the published results.
Many studies have reported the direct detection and identification
of bacterial pathogens from urine samples using MALDI-TOF MS

9. MATERIALS AND METHODS

10. Search strategy
They systematically searched
original papers published in
PubMed, Embase, Web of
Science, and the China National
Knowledge Infrastructure
(CNKI) databases.
These databases were queried
with the following keywords
and subject terms: “MALDI” or
“Matrix-assisted Laser
Desorption Ionization”, “urine”
or “urinary tract infections” or
“infection”, and “identification”.
Disagreements were resolved by
consultation with a third
researcher.
To obtain relevant studies, the
articles were first screened by
title and abstract; then, full
articles were further evaluated.
The search was limited to
publications in Chinese or
English. They searched the
databases for relevant articles
published till March 31, 2018.

11. Inclusion criteria
• Studies evaluating and comparing the accuracy of MALDI-TOF MS with that of
routine identification methods for the identification of clinical pathogens in
urinary tract infections were considered eligible for the meta-analysis.
• Typically, routine identification methods included the Vitek II system, API
system, MicroScan WalkAway system or other routine biochemical tests
and/or 16S rRNA sequencing.
• They contacted the authors for additional data for analysis if needed.

12. Exclusion criteria
• Studies that did not investigate urinary tract infections or studies lacking urine
specimens
• Studies lacking reference methods, a comparator method or gold standard
• Studies identifying clinical pathogens by mass spectrometry methods other
than MALDI-TOF MS.

13. Data extraction
• Two reviewers (MT and JY) independently extracted pertinent
data from each study.
• Any inconsistencies were resolved in a consensus meeting or
through discussion with the third author (YL).
• The quality of eligible studies was assessed by using the
Quality Assessment of Diagnostic Accuracy Studies
(QUADAS) 2 questionnaire.
Whiting PF., Rutjes AWS., Westwood ME., Mallett S., Deeks JJ., Reitsma JB., et al.,QUADAS-2: A Revised Tool for
the Quality Assessment of Diagnostic Accuracy Studies,Ann Intern Med ,2011,155(8): 529-536;DOI: 10.7326/0003-
4819- 155-8-201110180-00009

14. http://www.bristol.ac.uk/population-health-sciences/projects/quadas/

15. Data extraction
• The following data were included:
First author
Name
Year of
publication
Country
Number of
patients
MALDI-TOF
MS system
TP (true
positive), & FP
(false positive)
FN (false
negative), & TN
(true negative)
The method of
specimen
handling

16. Statistical analysis
• To obtain accurate and objective results, they defined the following
terms used to assess the sensitivity and specificity of MALDI-TOF
MS identification:
– True positive: the MALDI-TOF MS and culture results were consistent,
and only one microorganism was isolated in culture;
– False positive: the MALDI-TOF MS and culture results differed.
– False negative: a positive culture in the absence of MALDI-TOF MS
identification
– True negative: no MALDI-TOF MS identification of samples with a
negative culture.

17. Statistical analysis
• Analyses were performed using STATA version 12.0.
• The summary receiver operating characteristic (SROC) curve was
plotted to demonstrate a simultaneous non-linear relationship
between sensitivity and specificity and calculated to evaluate the
overall diagnostic performance of MALDI-TOF MS.
• Statistical heterogeneity was tested through the Q statistic and the
inconsistency index (I2) (a P value of Q<0.05 or I2 >50% was
considered the threshold indication of heterogeneity).
• The source of heterogeneity was further investigated by using
sensitivity analysis that ascertained whether the results were stable.

18. Statistical analysis
• Publication bias was determined by performing Deek’s funnel plot
asymmetry test. Statistical significance was defined as P < 0.05 or
an asymmetric funnel plot.

19. RESULTS

20. Systematic literature search

21. Study characteristics

22. Overall meta analysis
• Notably, current MALDI-TOF MS data software analysis is
unable to reliably identify all microorganisms present in a
mixture of microorganisms from direct samples.
(polymicrobial)
• Therefore, the authors calculated indexes after removing such
samples by the culture method.

24. Forest plot of the pooled sensitivity of
MALDI-TOF MS systems for identifying
pathogens.

25. Forest plot of the pooled specificity of
MALDI-TOF MS systems for identifying
pathogens.

26. Forest plot of positive likelihood ratio and negative likelihood ratio.

28. Performance of MS systems
• Further, they compared the diagnostic accuracy of the MALDI
Biotyper and the Vitek MS and found that
The estimated specificity and sensitivity of all
studies using the MALDI Biotyper were 0.94
(95% CI 0.75- 0.99) and 0.86 (95% CI 0.79-
0.91), respectively,
While the specificity and sensitivity of the
Vitek MS were 0.91 (95% CI 0.87-0.94) and
0.81 (95% CI 0.70-0.89), respectively.

29. Heterogeneity analysis and sensitivity analysis
There was substantial heterogeneity among the studies
(overall I2 for the bivariate model: 95%, 95% CI 91-99
and P<0.001)
Further, the source of heterogeneity was investigated by
inspecting pooled estimates that were calculated by
omitting one study at a time.
• There was no evidence that any individual study had an obvious
effect on the combined overall results.
• Hence, the results of this meta-analysis were stable.

30. Publication bias analysis
• Publication bias among the included studies was analyzed by
performing funnel plot asymmetry test using STATA software.
• The results indicated that there was no publication bias among
studies (P=0.15)

31. CONCLUSION

32. Conclusion
The results of the meta-analysis suggested that
MALDI-TOF MS could directly identify
microorganisms from urine samples with
higher sensitivity and specificity than routine
methods.
More studies should be conducted to confirm
the performance this technology and many
aspects of MALDI-TOF MS must be improved,
such as the ability to provide information about
the antimicrobial susceptibility of pathogens in
clinical samples.

33. CRITICAL APPRAISAL

35. Question Yes No Can’t Tell
Did the review address a clearly focused question?
Did the authors look for the right type of papers?
Do you think all the important, relevant studies were included?
Did the review’s authors do enough to assess quality of the
included studies?
If the results of the review have been combined, was it reasonable
to do so?
Can the results be applied to the local population?
Were all important outcomes considered?
. Are the benefits worth the harms and costs?

36. Critical Appraisal Summary
Positive Points Negative Points
Very important topic, first meta-analysis The studies enrolled in the meta-analysis were restricted only to articles
published in Chinese or English.
Effective papers search strategy Elimination of samples that were containing two or more pathogens,
which may have resulted in the overestimation of accuracy.
Defined criteria for study selection The number of included studies was small

37. References
• Flores-Mireles AL., Walker JN., Caparon M.,
Hultgren SJ., Urinary tract infections: epidemiology, mechanisms of infection and
treatment options,Nat Rev Microbiol,2015,13(5),269-284;
DOI:10.1038/nrmicro3432
• Lavigne, Jean-Philippe, Espinal, Paula, Dunyach-Remy, Catherine, Messad,
Nourredine, Pantel, Alix and Sotto, Albert. "Mass spectrometry: a revolution in
clinical microbiology?" Clinical Chemistry and Laboratory Medicine (CCLM),
vol. 51, no. 2, 2013, pp. 257-270. https://doi.org/10.1515/cclm-2012-0291
• Whiting PF., Rutjes AWS., Westwood ME., Mallett S., Deeks JJ., Reitsma JB., et
al.,QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic
Accuracy Studies,Ann Intern Med ,2011,155(8): 529-536;DOI: 10.7326/0003-
4819- 155-8-201110180-00009
• http://www.bristol.ac.uk/population-health-sciences/projects/quadas/

38. 38