Slides from a workshop delivered at The Ontario Public Health Convention 2014, entitled: "Making friends with systematic reviews".

1. Making friends with
systematic reviews:
Be comfortable assessing quality &
interpreting forest plots in 90 minutes!
March 31, 2014
Maureen Dobbins RN, PhD
Kara DeCorby, MSc
McMaster University

2. Disclosure of Commercial
Support
CFPC Conflict of Interest
Presenter Disclosure
Presenter: Maureen Dobbins
Relationships with commercial interests:
• Grants/Research Support: None
• Speakers Bureau/Honoraria: None
• Consulting Fees: None
• Other: None

3. Today’s Objectives
1. To refresh EIDM knowledge and skill
2. To become familiar with systematic
reviews
3. To identify the parts of a forest plot
4. To learn and practice how to read forest
plots
5. To increase comfort with critical appraisal
criteria and how to assess review quality
3
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

4. Model of EIDM in Public Health
4
Community
Health Issues,
Local Context
Research
Evidence
Public Health
Resources
Community and
Political Preferences
and Actions
Public Health
Expertise
National Collaborating Centre for Methods and Tools. (2012). A Model for Evidence‐Informed Decision‐Making in Public Health. [fact sheet].
Retrieved from http://www.nccmt.ca/pubs/FactSheet_EIDM_EN_WEB.pdf.
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

5. Evidence-informed
public health
5
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions
National Collaborating Centre for Methods and Tools. STEPS in Evidence‐Informed Public
Health in Evidence‐Informed Public Health. Retrieved February 10, 2013. From
http://www.nccmt.ca/eiph/index‐eng.html

6. Hierarchy of Evidence
6
Electronic systems, such as clinical
decision aids.
Outline of management options for a given health issue;
incorporate highest quality and most synthesized research.
Summarize the findings and implications of high
quality systematic reviews.
Rigorous summary of primary research relevant to
a particular focused question.
Brief summary of results and implications of
Single, high-quality studies.
Related to a particular focused question.
Systems
Single studies
Syntheses
Synopses of syntheses
Summaries
Synopses of single studies
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions
Dicenso, Alba, Bayley, Liz, & Haynes, R. Brian. (2009a). Accessing pre‐appraised evidence: fine‐tuning the 5S model into a 6S model. Evidence‐
Based Nursing, 12(4), 99‐101.
DiCenso, Alba, Bayley, Liz, & Haynes, R. Brian. (2009b). ACP Journal Club. Editorial: Accessing preappraised evidence: fine‐tuning the 5S model
into a 6S model. Annals of Internal Medicine, 151(6), JC3‐2, JC3‐3

7. Systematic Review
• A synthesis or summary of single
studies on a health topic
• A formalized (systematic), transparent,
scientific process
• Multiple reviewers involved
7
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

8. Steps of a systematic review
1. Stating the objectives of the research
2. Defining eligibility criteria for studies to be
included
3. Searching for (all) potentially eligible studies
4. Applying eligibility criteria
5. Assembling the most complete dataset feasible
6. Analyzing this dataset, using statistical synthesis
and sensitivity analyses, if appropriate and
possible
7. Preparing a structured report of the research.
8
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

9. Anatomy of a systematic review
• Introduction (background, objectives)
• Methods (search strategy, analytic approach)
• Results (studies included, data, statistical
results)
• Discussion (interpretation, applicability,
limitations, future research)
• Appendices (data analyses)
9
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

10. Ekeland,E., Heian,F., & Hagen,K.B. (2005). Can exercise improve self esteem in children and young people? A systematic review of randomised
controlled trials. British Journal of Sports Medicine, 39(11), 792‐798.
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

11. Appraising a systematic review
1. Clearly focused question?
2. Inclusion / exclusion criteria?
3. Thorough search?
4. Appropriate search timeframe?
5. Level of evidence described?
11
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

12. Appraising a review, continued
6. Quality of included evidence described?
7. Methods transparent?
8. Appropriate to combine?
9. Results weighted?
10.Conclusions believable?
12
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

13. Critical Appraisal
Do I trust the
findings and can I
use the research in
my setting?
13
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

14. Critical Appraisal
- Identify study type
- Identify CA tool
- Conduct appraisal
14
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

15. Critical Appraisal
Type of study (appropriate tool)
Summary/ Guideline
(AGREE tool)
Brouwers, M., et al. (2010)
Systematic review
(HE tool)
Health Evidence. (2013)
Single study
– RCT, cohort, case
control, qualitative
(CASP tools)
Public Health Resource Unit (2006)
15
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

16. Summary:
• What is EIDM?
• What is a systematic review?
• What are the processes and parts of a
review?
• How do I read a forest plot?
• How do I assess quality of a review (critical
appraisal)?
16
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

17. Questions?
17
Objectives -- EIDM – Systematic reviews -- Forest plots -- Critical Appraisal -- Summary -- Questions

18. Additional Resources
• HE Practice Tools:
http://www.healthevidence.org/practice-tools.aspx
• NCCMT Online Learning Modules:
http://www.nccmt.ca/modules/index-eng.html
– Intro to EIDM;
– Searching for Research Evidence in PH
18

19. References
• Brouwers, M., Kho, M.E., Browman, G.P., Burgers, J.S., Cluzeau, F., Feder, G., Fervers, B.,
Graham, I.D., Grimshaw, J., Hanna, S., Littlejohns, P., Makarski, J., Zitzelsberger, L. for the
AGREE Next Steps Consortium. (2010). AGREE II: Advancing guideline development,
reporting and evaluation in healthcare. Canadian Medical Association Journal, 182(18),
e839-e842. doi:10.1503/cmaj.090449
• Dicenso, Alba, Bayley, Liz, & Haynes, R. Brian. (2009a). Accessing pre-appraised evidence:
fine-tuning the 5S model into a 6S model. Evidence-Based Nursing, 12(4), 99-101.
• DiCenso, Alba, Bayley, Liz, & Haynes, R. Brian. (2009b). ACP Journal Club. Editorial:
Accessing preappraised evidence: fine-tuning the 5S model into a 6S model. Annals of
Internal Medicine, 151(6), JC3-2, JC3-3
• Ekeland,E., Heian,F., & Hagen,K.B. (2005). Can exercise improve self esteem in children and
young people? A systematic review of randomised controlled trials. British Journal of Sports
Medicine, 39(11), 792-798.
• Health Evidence. Quality assessment tool: Review articles.
http://www.healthevidence.org/documents/our-appraisal-
tools/QA_tool&dictionary_18.Mar.2013.pdf . 2013. McMaster Univerisity.
• National Collaborating Centre for Methods and Tools. (2012). A Model for Evidence-Informed
Decision-Making in Public Health. [fact sheet]. Retrieved from
http://www.nccmt.ca/pubs/FactSheet_EIDM_EN_WEB.pdf.
• National Collaborating Centre for Methods and Tools. STEPS in Evidence-Informed Public
Health in Evidence-Informed Public Health. Retrieved February 10, 2014 From
http://www.nccmt.ca/eiph/index-eng.html
• Public Health Resource Unit (2006). The Critical Skills Appraisal Programme: making sense
of evidence. Public Health Resource Unit, England. Retrieved from: http://www.casp-uk.net/19

20. Thank you!
Contact us:
dobbinsm@mcmaster.ca
kdecorby@healthevidence.org