The document discusses using a consequentialist approach and the GRADE method in health technology assessment (HTA) to evaluate diagnostic tests based on patient outcomes. It provides an example of using this approach to evaluate the appropriateness of FDG-PET in assessing early response to treatment for esophageal cancer. A panel of 60 experts from 7 tumor areas used the RAND/UCLA Appropriateness Method to rate 55 diagnostic questions. For 73% of questions, the panels followed the expected consequentialist approach of the GRADE method in their ratings.

1. Agenzia sanitaria e sociale regionale
Methods in HTA – Health outcomes
Bringing patients’ outcome at the forefront in HTA of
diagnostic tests using the GRADE approach
Luciana Ballini
Luca Vignatelli, Susanna Maltoni, Antonella Negro, Fabio Trimaglio, Roberto Grilli
Bilbao 26th June 2012
http://asr.regione.emilia-romagna.it

2. Appropriateness of
Agenzia sanitaria e sociale regionale
FDG-PET in oncology: 3rd update
Since 2001 a permanent research activity
alongside the development and diffusion
of the technology
An obsession ?
… or an opportunity
• To reason on diagnostic-therapeutic strategies, rather than simply on
diagnostic test
• To address research gaps (clinical effectiveness)
• To test a new methods for working groups developing diagnostic
recommendations

3. Appropriateness of a diagnostic test
Agenzia sanitaria e sociale regionale
The value of any medical test is ultimately measured by whether the
information it provides affects patient-relevant outcomes (Bossuyt 2010)
• Initial diagnostic assessment with subsequent
therapeutic approach
• The potential of the new test to modify initial diagnostic Rationale
assessment (e.g. stage of disease)
• The change in management following change of
diagnostic assessment
• The diagnostic accuracy of the new test Available evidence
• The resulting improvement in patients’
outcome due to change in therapeutic ?
approach induced by new test’s results
Schunemann et al BMJ 2008;336:1106-1110.

4. Agenzia sanitaria e sociale regionale Consequentialist Approach
NOT MODIFIABLE MODIFIABLE
Initial diagnostic
assessment
Treatment options
Change in Management effectiveness
SIGNIFICANT NEGLIGIBLE CURATIVE PALLIATIVE
Diagnostic Impact on
accuracy patients’
outcomes
Diagnostic Outcomes
efficacy trade off
TEST POSITION harms versus benefits
APPROPRIATENESS
CRITERIA

5. Agenzia sanitaria e sociale regionale
The
consequentialist
approach at work :
information
provided to the
panel

6. NOT MODIFIABLE MODIFIABLE
Initial diagnostic
assessment
Agenzia sanitaria e sociale regionale
Change in Management Treatment options
effectiveness
SIGNIFICANT NEGLIGIBLE CURATIVE PALLIATIVE
CLINICAL QUESTION:
Role of FDG-PET in early response to pre-operative chemoradiation of
patients treated for locally advanced esophageal cancer
Treatment effectiveness : Preoperative chemo/radio therapy improves 2
year survival by 13% (absolute difference) compared to surgical treatment
only (Gebski Lancet Oncology 2007)
Rationale: Preoperative chemo/radio therapy reduces tumour mass but increases risk
of post-operative mortality (ESMO 2010). Early identification of non responders could
avoid futile treatment and unnecessary risks
RESEARCH QUESTION: FDG-PET as replacement
Is FDG-PET accurate in evaluating early response to pre-operative chemoradiation
of patients treated for locally advanced esophageal cancer ?

7. Diagnostic
Outcomes
accuracy
importance
Agenzia sanitaria e sociale regionale
LEVEL OF EVIDENCE
PATIENT IMPORTANT OUTCOMES
Schunemann et al BMJ 2008;336:1106-1110.

8. Diagnostic Outcomes
efficacy trade off
harms versus benefits
TEST POSITION
Agenzia sanitaria e sociale regionale
APPROPRIATENESS
CRITERIA
Median scores
of importance
7
8
7
5
Not good enough
INAPPROPRIATE

9. RAND/UCLA Appropriateness Method: Results
Agenzia sanitaria e sociale regionale
7 panels (60 experts) – 7 tumours Lung
Breast
Head & Neck
55 Esophageal
questions
Colo-rectal
H Lymphoma
12
11 4
indeterminate
13
inappropriate
27
inappropriate NH Lymphoma
appropriate Disagreement
No rationale
5 9 11
appropriate inappropriate 2 uncertain
disagreement
28/55 (50.9%) resolved at first voting round 9 light
Disagreement
16/55 (29.1%) resolved at second voting round
11/55 (20%) persistent disagreement
2 Strong
Disagreement

10. Was the approach followed by the panels ?
Agenzia sanitaria e sociale regionale
21
20
18
N° of questions 16
14 YES (40 questions)
14
12 NO (4 questions)
10
8
6 4
4 2 1
2 1
1
0
in
te
ria
de
in
p in
te
pr o rta
ap
rm
e te
un
ap nc ri a
pr
in
u
ce
p te
op
pro
ap
ina
at
rt
ria
e
ap
pr
ai
Observed (RAND) in rm
et e
te
n
op
ind
ria
Expected(GRADE)
te
Yes in 73% of questions
(considering also the 11 unresolved questions)

11. Agenzia sanitaria e sociale regionale
The panelists
Thank you
luballini@regione.emilia-romagna.it http://asr.regione.emilia-romagna.it