My talk during the WAC 2015 in Seoul. What is the GRADE approach, why should we use clinical practice guidelines, how to understand recommendations using GRADE

1. Carlos A. Cuello Garcia
MD, PhD(c)
Health Research Methodology Program
Department of Clinical Epidemiology and
Biostatistics
Guideline
development
Levels of evidence
and the GRADE
approach

2. DISCLOSURE
• currently working at
MacGRADE centre &
Cochrane GRADEing group
• Cochrane author
• I have received support from
the WAO for travel / meetings
• no other COIs related to this
talk
GRADEing
Methods

3. OUTLINE
• what are clinical practice guidelines?
• why should we use them?
• what are recommendations and levels of
evidence?
• how to interpret a CPG recommendation?

5. ATOPIC
DERMATITIS

6. EASY!
PIMECROLIMUS

7. EASY!
PIMECROLIMUS
WHAT ABOUT CANCER
REPORTED?

8. WE BETTER STAY WITH
TOPICAL
CORTICOSTEROIDS

9. WE BETTER STAY WITH
TOPICAL
CORTICOSTEROIDS
PIMECROLIMUS IS
BETTER THAN TCS

10. ?
??

11. PIMECROLIMUS OR TOPICAL CORTICOSTEROIDS?
Meanwhile, at the Ministry of Health…

12. CLINICAL PRACTICE GUIDELINES
๏ statements that include recommendations
intended to optimize patient care that are
informed by a systematic review of evidence
and an assessment of the benefits and harms
of alternative care options.
IOM · AHRQ 2011

13. CLINICAL
RECOMMENDATION
EVIDENCE
PATIENT
VALUES
ACCEPTA-
BILITY
BENEFITS
VS HARMS
COSTS
IN PATIENTS (CLINICAL)
GUIDELINES

14. DECISION
EVIDENCE
PATIENT
VALUES
ACCEPTA-
BILITY
BENEFITS
VS HARMS
FEASIBILITY
EQUITY
RESOURCE
USE
IN PUBLIC HEALTH
GUIDELINES

15. guidelines we can trust
๏ Based on a
systematic review of
the existing evidence
1

16. guidelines we can trust
๏ Developed by a
knowledgeable,
multidisciplinary
panel of experts and
representatives from
key affected groups
2

17. guidelines we can trust
๏ Considers important
patient subgroups
and patient
preferences
3

18. guidelines we can trust
๏ Based on an explicit
and transparent
process that
minimizes distortions,
biases, and conflicts
of interest
4

19. guidelines we can trust
๏ Provides a clear
explanation of the logical
relationships between
alternative care options and
health outcomes, and
provide ratings of both the
quality of evidence and the
strength of
recommendations
5

20. guidelines we can trust
๏ It is revised as
appropriate when
important new
evidence warrants
modifications of
recommendations.
6

21. CLINICAL
RECOMMENDATION
or
DECISION
EVIDENCE
PATIENT
VALUES
ACCEPTA-
BILITY
BENEFITS
VS HARMS
FEASIBILITY
EQUITY
RESOURCE
USE

22. CLINICAL
RECOMMENDATION
or
DECISION
EVIDENCE

23. RESEARCH
GENERATION
PUBLISHING

24. RESEARCH
GENERATION
PUBLISHING

25. RESEARCH
GENERATION
PUBLISHING

26. RESEARCH
GENERATION
PUBLISHING
systematic
reviews
clinical practice
guidelines

27. RESEARCH
GENERATION
PUBLISHING
systematic
reviews
clinical practice
guidelines

28. RESEARCH
GENERATION
PUBLISHING
systematic
reviews
clinical practice
guidelines

29. CHALLENGES AHEAD
why we need
trustworthy clinical
practice guidelines?

30. ๏ Every year $100
billion dollars are
invested in
biomedical
research
๏ Only 10% is used to
test treatments
Chalmers 2009
$

31. ๏ only half of
researchers use a
systematic review
that has been
previously done on
their topic
Chalmers I, Glasziou P. Lancet 2009;374:86-9
Cooper NJ, et al Clin Trials 2005;2:260–4.
50%

32. ๏ only half of
researchers use
adequate tools to
ensure the quality
of their work
Chalmers I, Glasziou P. Lancet 2009;374:86-9
Cooper NJ, et al Clin Trials 2005;2:260–4.
CONSORT

33. ๏ not all research is
registered,
reported, or
published…
Cressey D. Secrets of trial data revealed.
Nature. 2013 Oct 10;502(7470):154-5
RESEARCH
GENERATION

34. and many times we overlook research
priorities and patient important
outcomes
Chalmers I, Glasziou P. Lancet 2009;374:86-9
Cooper NJ, et al Clin Trials 2005;2:260–4.

35. 100
participants
parachute placebo

36. 100
participants
36.5 (2.1) 36.7 (1.9)
4.3 (1.2) 6.1(1.3)
25 (3) 22(2.4)
Temp ºC –mean
(SD)
Fear scale –mean
(IQR)
Serum adrenaline
(mcg/dL) – mean
(SD)
parachute placebo

37. 100
participants
36.5 (2.1) 36.7 (1.9)
4.3 (1.2) 6.1(1.3)
25 (3) 22(2.4)
Temp ºC –mean
(SD)
Fear scale –mean
(IQR)
Serum adrenaline
(mcg/dL) – mean
(SD)
p=0.00001
parachute placebo

38. antiarrhythmics
Patients
with MI
ecainide
flecainide
placebo
↓ ↓
↑↑
arrhythmias
death ??
Epstein AE, et al. JAMA 1993

39. outcomes Patients with
asthma
nitrous
oxide
placebo
O2 sat
FEV1
Length of hospital stay
Death

40. so, do you recommend..?

41. so, do you recommend..?
๏ pimecrolimus or topical
corticosteroids for atopic dermatitis?

42. so, do you recommend..?
๏ pimecrolimus or topical
corticosteroids for atopic dermatitis?
๏ nitrous oxide for severe asthma?

43. so, do you recommend..?
๏ pimecrolimus or topical
corticosteroids for atopic dermatitis?
๏ nitrous oxide for severe asthma?
๏ probiotics for allergy prevention?

44. so, do you recommend..?
๏ pimecrolimus or topical
corticosteroids for atopic dermatitis?
๏ nitrous oxide for severe asthma?
๏ probiotics for allergy prevention?
๏ epinephrine for anaphylactic shock?

45. expert recommendations
cross-sectional
case series, case reports
case-control
cohort
Randomized trial
Systematic review
CTFPHE
1979

46. expert recommendations
cross-sectional
case series, case reports
case-control
cohort
Randomized trial
Systematic review
CTFPHE
1979
I
good
II
fair
III
poor
EVIDENCE

47. expert recommendations
cross-sectional
case series, case reports
case-control
cohort
Randomized trial
Systematic review
CTFPHE
1979
I
good
II
fair
III
poor
EVIDENCE
A
B
C
RECOMMENDATION
D

48. I want to jump from this
airplane

49. do you recommend a
parachute?

51. Grading of Recommendations Assessment,
Development and Evaluation

52. expert
recommendations
cross-sectional
case series
case-control
cohort
RCT

53. cross-sectional
case series
case-control
cohort
RCT RCT
Observational
experience is
not evidence,
it’s a tool

54. By default...
cross-sectional
case series
case-control
cohort
RCT RCT
Observational
HIGH QUALITY
LOW QUALITY

55. QUALITY OF EVIDENCE
๏ You can also call it:
๏ certainty of evidence
๏ confidence in the estimates
Balshem H, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401-6

56. QUALITY OF EVIDENCE
Balshem H, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401-6
SYSTEMATIC REVIEW
CLINICAL PRACTICE
GUIDELINE
the extent of our confidence
that the estimates of the
effect are correct
the extent of our confidence
that the estimates of the
effect are adequate to
support a particular decision
or recommendation

57. QUALITY OF EVIDENCE
HIGH
We are very confident that the true effect lies close to
that of the estimate of the effect
We are moderately confident in the effect estimate: The
true effect is likely to be close to the estimate of the
effect, but there is a possibility that it is substantially
different
Our confidence in the effect estimate is limited: The
true effect may be substantially different from the
estimate of the effect
We have very little confidence in the effect estimate:
The true is likely to be substantially different from the
estimate of effect
MODERATE
LOW
VERY LOW

58. how is the quality (confidence)
of the evidence determined?

59. what would make you loose
confidence?
๏ Overall, probiotics reduced
the risk of eczema in
infants when given to their
pregnant mothers
๏ RR=0.72 (95% CI 0.61 to
0.85)
๏ 3’509 participants, 15
trials

60. QUALITY OF EVIDENCE
risk of
bias
STUDY 1
STUDY 2
STUDY 3

61. QUALITY OF EVIDENCE
inconsistencyrisk of
bias
STUDY 1
STUDY 2
STUDY 3

62. QUALITY OF EVIDENCE
inconsistency
indirectness
P
I
C
O
?
risk of
bias
STUDY 1
STUDY 2
STUDY 3

63. QUALITY OF EVIDENCE
inconsistency
indirectness
P
I
C
O
?
imprecision
risk of
bias
STUDY 1
STUDY 2
STUDY 3

64. QUALITY OF EVIDENCE
inconsistency
indirectness
P
I
C
O
?
imprecision
publication
bias
risk of
bias
STUDY 1
STUDY 2
STUDY 3

65. By default...
cross-sectional
case series, etc.
case-control
cohort
RCT RCT
Observational
HIGH QUALITY
LOW QUALITY

66. By default...
RCT
Observational
HIGH QUALITY
LOW QUALITY

67. RCT
HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE

68. RCT
HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE
risk of
bias

69. HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE
RCT
risk of
bias

70. HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE
RCT
risk of
bias
inconsistency

71. HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE
RCT
risk of
bias
inconsistency
imprecision

72. HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE
RCT
risk of
bias
inconsistency
imprecision
indirectness

73. HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE
RCT
risk of
bias
inconsistency
imprecision
indirectness
publication
bias

74. VERY LOW
QUALITY OF EVIDENCE
RCT
risk of
bias
inconsistency
imprecision
indirectness
publication
bias

75. OBSERVATIONAL
HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE

76. OBSERVATIONAL
HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE
strong
association

77. HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE
OBSERVATIONAL
strong
association
dose
response

78. HIGH
MODERATE
LOW
VERY LOW
QUALITY OF EVIDENCE
OBSERVATIONAL
strong
association
dose
response
opposingresidual
confounding

79. HIGH
QUALITY OF EVIDENCE
OBSERVATIONAL
strong
association
dose
response
opposingresidual
confounding

80. from evidence to
recommendations

81. Quality of evidence
Strength of a
recommendation
the extent of our confidence
that the estimates of an effect
are adequate to support a
particular decision or
recommendation
the extent to which one can be
confident that the desirable
consequences of an
intervention outweigh its
undesirable consequences

82. NO
YES
YES
NO
strengthofarecommendation direction of a recommendation
for
against

83. NO
YES
YES
NO
strengthofarecommendation direction of a recommendation
for
against
STRONG

84. NO
YES
YES
NO
strengthofarecommendation direction of a recommendation
for
against
CONDITIONAL
STRONG

85. patients
clinicians
policy makers
Most individuals in this
situation would want
the recommended
course of action, and
only a small proportion
would not
Most individuals should
receive the intervention.
Adherence to this
recommendation could
be used as a quality
criterion or
performance indicator
The recommendation
can be adopted as
policy in most
situations.
STRONG CONDITIONAL
The majority of individuals in
this situation would want the
suggested course of action,
but many would not
Different choices will be
appropriate for individual
patients. Must help each
patient arrive at a
management decision
consistent with his or her
values and preferences.
Policy making will require
substantial debate and
involvement of various
stakeholders.

86. FRAMEWORKS
evidence to recommendation

87. just
4 columns
&
conclusions

88. CRITERIA

89. CRITERIA
Problem
Quality of evidence
Benefits & harms
Values
Resource use
Equity
Acceptability
Feasibility

90. CRITERIA
Problem
Quality of evidence
Benefits & harms
Values
Resource use
Equity
Acceptability
Feasibility
JUDGEMENTS
RESEARCH
EVIDENCE
ADDITIONAL
INFORMATION

91. CRITERIA
Problem
Quality of evidence
Benefits & harms
Values
Resource use
Equity
Acceptability
Feasibility
JUDGEMENTS
RESEARCH
EVIDENCE
ADDITIONAL
INFORMATION
what is to be
considered?

92. CRITERIA
Problem
Quality of evidence
Benefits & harms
Values
Resource use
Equity
Acceptability
Feasibility
JUDGEMENTS
RESEARCH
EVIDENCE
ADDITIONAL
INFORMATION
what do we
consider about
it?

93. CRITERIA
Problem
Quality of evidence
Benefits & harms
Values
Resource use
Equity
Acceptability
Feasibility
JUDGEMENTS
RESEARCH
EVIDENCE
ADDITIONAL
INFORMATION
what do we
consider about
it?

94. CRITERIA
Problem
Quality of evidence
Benefits & harms
Values
Resource use
Equity
Acceptability
Feasibility
JUDGEMENTS
RESEARCH
EVIDENCE
ADDITIONAL
INFORMATION
based on what
evidence?

95. CONCLUSIONS
Balance of consequences
Decision /recommendation
Justification
Implementation considerations
Monitoring
Evaluation
Research priorities

96. example

97. Explanations Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013
Generic EtD framework 1
Evidence to decision framework
Question 1: Should probiotics vs. no probiotics be used in pregnant women?
Population:!pregnant!women!
Option:!probiotics!
Comparison:!no!probiotics!
Setting:3outpatient!
Perspective:3individual!patient
Background: The$intestinal$microbiome$could$play$an$important$role$in$the$immune$system$maturation,$
and$it$has$been$suggested$that$early6life$probiotic$administration,$whether$directly$to$the$infant$or$in$
their$mothers$breast$milk,$may$reduce$the$risk$of$allergies$in$childhood.$The$objective$of$this$question$is$
to$evaluate$the$impact$of$probiotics$administered$to$the$expecting$mothers$on$their$infant.
Subgroup considerations: subpopulation of women at high risk for allergy in a child
CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION
PROBLEM
Is the
problem a
priority?
No Probably no Uncertain Probably yes Yes
X
Allergic diseases represent a spectrum of health conditions and a worldwide burden in different
populations. (1)
Are a large
number of
people
affected?
No Probably no Uncertain Probably yes Yes
X
As many as 40% of the worldwide population is affected by any type of allergy. In infants
prevalence depends highly on the allergic status of their parents, being approximately of 10% in
those without an allergic parent or sibling, versus 20% to 30% in those with the atopic
background in their relatives. (2)
Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013

98. Explanations Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013
Generic EtD framework 2
CRITERIA JUDGEMENTS RESEARCH EVIDENCE
ADDITIONAL
CONSIDERATIONS
VALUES
Is there
important
uncertainty
or variability
about how
much people
value the
main
outcomes?
Important
uncertainty
or
variability
Possibly
important
uncertainty
or
variability
Probably
no
important
uncertainty
or
variability
No
important
uncertainty
or
variability
No known
undesirable
outcomes
X
Detailed judgements
The relative importance or values of the main outcomes of interest:
Outcome Relative importance Certainty of the evidence
Eczema critical low
Asthma/wheezing critical low
Food allergy critical low
Adverse effects critical low
We judged that the outcomes
eczema, asthma and food
allergy are critical for people.
The adverse outcomes are
probably of high importance
and the burden of taking daily
pills is limited. Some
immunocompromised women
might not accept the risk.
BENEFITS&HARMSOFTHEOPTIONS
What is the
overall
certainty of
the evidence
of
effectiveness?
No
included
studies Very low Low Moderate High
X
Summary of findings:
Outcome With
[intervention]
Without
[intervention]
Difference
(per 100)
(95%CI)
Relative
effect (RR)
(95%CI)
Certainty
of the
evidence
(GRADE)
Eczema
(follow-up 1 to 5 years)
365/1520
(24%)
484/1515
(31.9%)
9 fewer per
100 (from 4
fewer to 13
fewer)
RR 0.72 (0.6 to
0.86)
⊕⊝⊝⊝
VERY LOW
Asthma/wheezing
(follow-up 2 to 7 years)
143/992
(14.4%)
139/982
(14.2%)
0 fewer per
100 (from 3
fewer to 3
more)
RR 0.97 (0.77
to 1.22)
⊕⊕⊝⊝
LOW
Food allergy
(follow-up 1 to 2 years)
36/279
(12.9%)
41/284
(14.4%)
1 more per
100 (from 3
fewer to 8
more)
RR 1.08 (0.73
to 1.59)
⊕⊝⊝⊝
VERY LOW
Adverse effects 101/394
(25.6%)
88/397
(22.2%)
3 more per
100 (from 4
fewer to 12
more)
RR 1.13 (0.82
to 1.52)
⊕⊝⊝⊝
VERY LOW
Link to detailed evidence profile
Subgroup considerations:
Link(s) to summary of findings and judgments for subgroups
The data are indirect for all
outcomes because they are
primarily derived from studies
that looked at mixed exposure
in women during pregnancy
and breastfeeding and of
infants after birth. Only 1 RCT
assessed the effect on
eczema in pregnant women
only: RR 0.88 (0.63 to 1.22);
RD 5 fewer per 100 (from 14
fewer to 9 more)
5 RCTs included pregnant
women + later breastfeeding
mothers: RR 0.5 (0.4 to 0.63);
RD 21 fewer per 100 (from 15
fewer to 25 fewer)
5 RCTs included pregnant
women + infants after
birth (follow-up 1 to 5 years):
RR 0.87 (0.72 to 1.04), RD 4
fewer per 100 (from 8 fewer
to 1 more)
3 RCTs included pregnant
women + subsequently
breastfeeding +
infants (follow-up 3 to 4
years): RR 0.78 (0.49 to
1.24); RD 7 fewer per 100
(from 17 fewer to 8 more)
No effects were observed on
asthma/wheezing and food
allergy.
How
substantial
are the
desirable
anticipated
effects?
Don’t
know
Not
important
Somewhat
important
Moderately
important
Very
important
Varies
X X
Detailed judgements
There was some disagreement among panel members
whether the effect is somewhat or moderately important.
How
substantial
are the
undesirable
anticipated
effects?
Don’t
know
Very
important
Moderately
important
Somewhat
important
Not
important
Varies
X
Detailed judgements
No serious adverse effects, and no difference in mild
adverse effects between the groups.
Do the
desirable
effects
outweigh the
undesirable
effects?
No Probably
No
Don’t know Probably
Yes
Yes Varies
X
Detailed judgements
Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013

99. Explanations Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013
Generic EtD framework 2
CRITERIA JUDGEMENTS RESEARCH EVIDENCE
ADDITIONAL
CONSIDERATIONS
VALUES
Is there
important
uncertainty
or variability
about how
much people
value the
main
outcomes?
Important
uncertainty
or
variability
Possibly
important
uncertainty
or
variability
Probably
no
important
uncertainty
or
variability
No
important
uncertainty
or
variability
No known
undesirable
outcomes
X
Detailed judgements
The relative importance or values of the main outcomes of interest:
Outcome Relative importance Certainty of the evidence
Eczema critical low
Asthma/wheezing critical low
Food allergy critical low
Adverse effects critical low
We judged that the outcomes
eczema, asthma and food
allergy are critical for people.
The adverse outcomes are
probably of high importance
and the burden of taking daily
pills is limited. Some
immunocompromised women
might not accept the risk.
BENEFITS&HARMSOFTHEOPTIONS
What is the
overall
certainty of
the evidence
of
effectiveness?
No
included
studies Very low Low Moderate High
X
Summary of findings:
Outcome With
[intervention]
Without
[intervention]
Difference
(per 100)
(95%CI)
Relative
effect (RR)
(95%CI)
Certainty
of the
evidence
(GRADE)
Eczema
(follow-up 1 to 5 years)
365/1520
(24%)
484/1515
(31.9%)
9 fewer per
100 (from 4
fewer to 13
fewer)
RR 0.72 (0.6 to
0.86)
⊕⊝⊝⊝
VERY LOW
Asthma/wheezing
(follow-up 2 to 7 years)
143/992
(14.4%)
139/982
(14.2%)
0 fewer per
100 (from 3
fewer to 3
more)
RR 0.97 (0.77
to 1.22)
⊕⊕⊝⊝
LOW
Food allergy
(follow-up 1 to 2 years)
36/279
(12.9%)
41/284
(14.4%)
1 more per
100 (from 3
fewer to 8
more)
RR 1.08 (0.73
to 1.59)
⊕⊝⊝⊝
VERY LOW
Adverse effects 101/394
(25.6%)
88/397
(22.2%)
3 more per
100 (from 4
fewer to 12
more)
RR 1.13 (0.82
to 1.52)
⊕⊝⊝⊝
VERY LOW
Link to detailed evidence profile
Subgroup considerations:
Link(s) to summary of findings and judgments for subgroups
The data are indirect for all
outcomes because they are
primarily derived from studies
that looked at mixed exposure
in women during pregnancy
and breastfeeding and of
infants after birth. Only 1 RCT
assessed the effect on
eczema in pregnant women
only: RR 0.88 (0.63 to 1.22);
RD 5 fewer per 100 (from 14
fewer to 9 more)
5 RCTs included pregnant
women + later breastfeeding
mothers: RR 0.5 (0.4 to 0.63);
RD 21 fewer per 100 (from 15
fewer to 25 fewer)
5 RCTs included pregnant
women + infants after
birth (follow-up 1 to 5 years):
RR 0.87 (0.72 to 1.04), RD 4
fewer per 100 (from 8 fewer
to 1 more)
3 RCTs included pregnant
women + subsequently
breastfeeding +
infants (follow-up 3 to 4
years): RR 0.78 (0.49 to
1.24); RD 7 fewer per 100
(from 17 fewer to 8 more)
No effects were observed on
asthma/wheezing and food
allergy.
How
substantial
are the
desirable
anticipated
effects?
Don’t
know
Not
important
Somewhat
important
Moderately
important
Very
important
Varies
X X
Detailed judgements
There was some disagreement among panel members
whether the effect is somewhat or moderately important.
How
substantial
are the
undesirable
anticipated
effects?
Don’t
know
Very
important
Moderately
important
Somewhat
important
Not
important
Varies
X
Detailed judgements
No serious adverse effects, and no difference in mild
adverse effects between the groups.
Do the
desirable
effects
outweigh the
undesirable
effects?
No Probably
No
Don’t know Probably
Yes
Yes Varies
X
Detailed judgements
Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013

100. Explanations Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013
Generic EtD framework 3
CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS
RESOURCEUSE
How large are
the resource
requirements?
Large
costs
Moderate
costs
Small Moderate
savings
Large
savings
Varies
X
Detailed judgements
Prices are likely to vary substantially depending on the
setting. This may be a particularly important
consideration in low and middle-income countries. A
level and type of insurance may play a substantial role
as well.
From a health systems point of view it might also be cost
effective given that probiotic would be used for 9 months
and cost of treatment of eczema may be distributed
across many years.
Extremely limited research evidence (internet searches of drug prices)
Bifidobacterium
bifidum (cost per
person per year
US$) Dose: 1 pill
each day
Lactobacillus
gg (cost per
person per
year US$) 1
pill each day
North-
America
Average $181.16 $341.6
South-
America
Average $174.3 $286
Europe
Average $167.86 $251.56
Fewer office visits would occur as a result of eczema if the
effects on eczema were true.
How large is
the
incremental
cost relative
to the net
benefit?
Very
large
ICER
Large
ICER
Moderate
ICER
Small
ICER
Savings Varies
Detailed judgements
No research evidence
If eczema was reduced the intervention might be cost-
effective given fewer office visits (between $17,400 and
$34,100 to treat 100 people for 1 year or ¾ of that for 9
months) preventing 9 cases of eczema.
In most studies probiotics were used in the last trimester of
pregnancy, which, if used this same way, might reduce the
cost per pregnant woman.
EQUITY
What would
be the impact
on health
inequities?
Increased Probably
increased
Uncertain Probably
reduced
Reduced Varies
X
Detailed judgements
No research evidence
In some settings it may be important to consider equity as
the access may depend on socioeconomic status of the
country or setting where coverage will depend on
policymakers.
ACCEPTABILITY
Is the option
acceptable
to key
stakeholders?
No Probably
No
Uncertain Probably
Yes
Yes Varies
X
Detailed judgements
No research evidence
Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013

101. Explanations Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013
Generic EtD framework 4
CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS
FEASIBILITY
Is the option
feasible to
implement?
No Probably
No
Uncertain Probably
Yes
Yes Varies
X
Detailed judgements
No research evidence
Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013

102. Explanations Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013
Generic EtD framework 5
Recommendation Should probiotics vs. no probiotics be used in pregnant women (exposing their children in utero)?
Overall balance of consequences Undesirable consequences
clearly outweigh desirable
consequences
Undesirable consequences
probably outweigh desirable
consequences
The balance of desirable and
undesirable consequences indicates
they are very similar*
Desirable consequences
probably outweigh undesirable
consequences
Desirable
consequences
clearly outweigh
undesirable
consequences
! ! ! X !
We recommend against the
option or for the alternative
We suggest not to use the option or to use the
alternative
We suggest using the option We recommend the
option
! ! X !
Panel decisions 3 panel members with potential COI recused themselves from participating in formulating the recommendation. Consensus was obtained from the rest of the team.
Recommendation (text)
The guideline panel suggests using probiotics in pregnant women at high risk for allergy in their children (conditional recommendation, very low quality
evidence).
Remarks and justification Most studies commenced probiotics in the last trimester of pregnancy.
The very low quality evidence for adverse effects indicates that our confidence in the absence of increased adverse effects is low. Future research is needed (see
definitions of very low quality) e.g., generalizing to immune-compromised children
Subgroup considerations Women with high risk of allergy in their children
Women with average risk of allergy in their children
Implementation considerations This recommendation is based on trials investigating the probiotics or mixtures of probiotic listed below. We have not found a difference between these different
probiotics, but that does not mean there is no difference
Monitoring and evaluation
considerations
Research priorities Develop instruments for evaluating the risk of allergy in children as the family history predicts only about 30% of the population risk. There is some evidence that first child
is at higher risk for allergy than subsequent children. Long-term follow-up of long-term effects.
No direct evidence for the use of probiotics in formula – this should be evaluated in future research and is an unmet need.
Prepared by: Holger, Jan, Carlos, Juan
Date: December 10, 2013

103. OUTLINE
• what are clinical practice guidelines?
• why should we use them?
• what are recommendations and levels of
evidence?
• how to interpret a CPG recommendation?

104. tools / more readings
• guidelinedevelopment.org
• cebgrade.mcmaster.ca
• gradeing.cochrane.org
GRADEing
Methods

105. Thank you!
cuelloca@mcmaster.ca
@CharlieNeck