3. Narrative reviews
• Usually written by experts in the field
• Use informal and subjective methods to collect and interpret
information
• Usually narrative summaries of the evidence
Systematic Review
• A review of the evidence on a clearly formulated question that uses
systematic and explicit methods to identify, select and critically
appraise relevant primary research, and to extract and analyse data
from the studies that are included in the review*
Read: Klassen et al. Guides for Reading and Interpreting Systematic
Reviews. Arch Pediatr Adolesc Med 1998;152:700-704.
4. Key elements of a systematic review
Structured, systematic process involving several
steps :
1. Formulate the question
2. Plan the review
3. Comprehensive search
4. Unbiased selection and abstraction process
5. Critical appraisal of data
6. Synthesis of data (may include meta-analysis)
7. Interpretation of results
All steps described explicitly in the review
5. Systematic vs. Narrative reviews
• Scientific approach to a
review article
• Criteria determined at outset
• Comprehensive search for
relevant articles
• Explicit methods of appraisal
and synthesis
• Meta-analysis may be used to
combine data
• Depend on authors’
inclination (bias)
• Author gets to pick any
criteria
• Search any databases
• Methods not usually
specified
• Vote count or narrative
summary
• Can’t replicate review
6. Advantages of systematic reviews
• Reduce bias
• Replicability
• Resolve controversy between conflicting studies
• Identify gaps in current research
• Provide reliable basis for decision making
Limitations of systematic reviews specific to health
promotion
• Results may still be inconclusive
• There may be no trials/evidence
• The trials may be of poor quality
• The intervention may be too complex to be tested by a trial
• Practice does not change just because you have the evidence of
effect/effectiveness
7. Consider these interventions…
•Interventions to promote smoke alarm ownership and
function
•School-based driver education for the prevention of
traffic crashes
•Helmets for preventing head and facial injuries in
bicyclists
Do you think the results identified in SRs will be good,
promising or absent (and potentially harmful)?
8. Results from systematic reviews
• Helmets reduce bicycle-related head and facial injuries for
bicyclists of all ages involved in all types of crashes including
those involving motor vehicles.
• The results provide no evidence that drive education
reduces road crash involvement, and suggest that it may
lead to a modest but potentially important increase in the
proportion of teenagers involved in traffic crashes.
• Results from this review suggest that area-wide traffic
calming in towns and cities may be a promising
intervention for reducing the number of road traffic injuries
and deaths. However, further rigorous evaluations of this
intervention are needed.
9. The Cochrane Collaboration
International non-profit
organisation that prepares,
maintains, and disseminates
systematic up-to-date reviews
of health care interventions
10. Cochrane Collaboration
Named in honour of Archie Cochrane, a British
researcher
In 1979:
“It is surely a great criticism
of our profession that we
have not organised a critical
summary, by specialty or
subspecialty, adapted
periodically, of all relevant
randomised controlled trials”
11. The Cochrane Library
• Cochrane Systematic reviews : Cochrane reviews and protocols
• Database of Reviews of Effects: Other systematic reviews appraised by
the Centre for Reviews and Dissemination.
• Cochrane Central Register of Controlled Trials:
Bibliography of controlled trials (some not indexed in MEDLINE).
• Health Technology Assessment Database: HTA reports
• NHS Economic evaluation database:
Economic evaluations of health care interventions.
13. Questions of interest
Effectiveness:
•Does the intervention work/not work?
•Who does it work/not work for?
Other important questions:
•How does the intervention work?
•Is the intervention appropriate?
•Is the intervention feasible?
•Is the intervention and comparison relevant?
16. An answerable question
Q. Are mass media (or school-based or community-
based) interventions effective in preventing smoking in
young people?
17. Problem,
population
Intervention Comparison Outcome Types of
studies
Young people
under 25 years
of age
a) Television
b) Radio
c) Newspapers
d) Bill boards
e) Posters
f) Leaflets
g) Booklets
a) School-based
interventions
b) No
intervention
a) objective
measures of
smoking (saliva
thiocyanate
levels, alveolar
CO)
b) self-reported
smoking
behaviour
c) Intermediate
measures
(intentions,
attitude,
knowledge,
skills)
d) Media reach
a) RCT
b) Controlled
before and after
studies
c) Time series
designs
The PICO(T) chart
19. Systematic review process
1. Well formulated question
2. Comprehensive data search
3. Unbiased selection and abstraction process
4. Critical appraisal of data
5. Synthesis of data
6. Interpretation of results
20. A good search
Clear research question
Comprehensive search
• All domains, no language restriction, unpublished and published literature,
up-to-date
Document the search (replicability)
21. Components of electronic searching
1. Describe each PICO component
2. Start with primary concept
3. Find synonyms
a) Identify MeSH / descriptors / subject headings
b) Add textwords
4. Add other components of PICO question to
narrow citations (may use study filter)
5. Examine abstracts
6. Use search strategy in other databases (may
need adapting)
34. Critical appraisal
The process of systematically
examining research evidence to
assess its validity, results and
relevance before using it to inform a
decision.
Alison Hill, Critical Appraisal Skills Programme, Institute of Health
Sciences, Oxford http://www.evidence-based-medicine.co.uk
36. Why appraise validity?
•Not all published and unpublished literature is
of satisfactory methodological rigour
• Just because it is in a journal does not mean it is
sound!
• Onus is on you to assess validity!
•Quality may be used as an explanation for
differences in study results
•Guide the interpretation of findings and aid in
determining the strength of inferences
40. Appraisal of a systematic review
• 10 questions
1. Clearly-focused question
2. The right type of study included
3. Identifying all relevant studies
4. Assessment of quality of studies
5. Reasonable to combine studies
6. What were the results
7. Preciseness of results
8. Application of results to local population
9. Consideration of all outcomes
10. Policy or practice change as a result of evidence
CASP