This document summarizes a study comparing healthcare technology assessment (HTA) recommendations across five countries (England, Scotland, Sweden, Canada, Australia) and three therapy areas (cancer, orphan diseases, central nervous system diseases). The study found low levels of agreement between HTA agencies and differing associations between agencies, recommendations, and therapy areas. Differences in recommendations were attributed to national considerations and differences in HTA processes, such as evidence used, methods, and interpretation. Understanding why recommendations vary can help improve HTA methodologies and determine when differences reflect legitimate national priorities versus inconsistent application of HTA principles.

1. A COMPARATIVE ANALYSIS OF HTA OUTCOMES IN FIVE
COUNTRIES ACROSS THREE THERAPY AREAS
Cancer, orphan and central nervous system diseases
Authors: Elena Nicod, Dr Panos Kanavos
London School of Economics and Political Science
LSE Health and Social Care
HTAI Conference
Bilbao, 26 June 2012

2. Agenda
• Background and objectives
• Methods
• Results
• Discussion & Policy implications

3. Background & objectives

4. Objectives: Examine and compare HTA recommendations in 5 countries, and
across 3 therapy areas (cancer, orphan, central nervous system diseases)
Conceptual framework: It is assumed that HTA outcomes are likely to be similar
across settings, because the same clinical and safety evidence is considered
alongside the same methodology
In practice: HTA recommendations vary greatly across settings1, 2, 3, 4
Implications
Coverage decisions may vary across countries in the same geographical region, with
the same socio-economic level of development, and the same health care systems
Need to understand why these differences across countries occur
Need to approximate HTA processes across settings
Sources: 1. Clement 2010; 2. Morgan 2006; 3. Lexchin 2008; 4. Barbieri 2005.

5. Methods

6. DATA
• Database: all drug-indication pairs issued between January 2007 and December 2009
• Country selection: England, Scotland, Sweden, Canada, Australia
• Stratification per disease area: WHO ICD10 codes
• HTA recommendation classification: List (“L”), List with Conditions (“LWC”), Do not list
(“DNL”)
• Selection of disease areas: cancer, orphan, central nervous system indications
ANALYSIS
• Comparison of HTA recommendations across the study countries
• Measurement of the level of agreement between the study countries: Cohen’s Kappa
score
• Associations between HTA recommendations, HTA bodies, and 3 therapy areas (cancer,
orphan, central nervous system diseases): two-way correspondence analysis
• Case studies: to understand possible reasons for variations

7. Results

8. Database - summary
= 287 drug-indication pairs collected between 2007-2009 in the 5 study countries
Figure represents the HTA recommendations issued in each country
Scotland
SMC 54 78 61
n=193
Sweden
TLV 79 26 6
n=111
Australia
PBAC 47 111 52
n=210
England
NICE 21 70 19
n=110
Canada
CDR/CED 4 62 64
n=130
0% 20% 40% 60% 80% 100%
Source: the authors
List LWC DNL

9. Snapshot of our database - orphan therapies
Drug Indication ICD10 Canada England Australia Sweden Scotland
CDR/CED NICE PBAC TLV SMC
Dasatinib Chronic myeloid leukemia C92.1 LWC Ongoing LWC L LWC
Sorafenib tosylate Renal cell carcinoma C64 DNL DNL DNL L DNL
Sorafenib tosylate Hepatocellular carcinoma C18-C21 LWC DNL LWC LWC DNL
Ambrisentan Pulmonary arterial hypertension I27 LWC LWC LWC LWC
Imatinib mesylate Chronic myeloid leukaemia C LWC LWC L LWC
Imatinib mesylate GIST C16-18 LWC LWC L DNL
Lenalidomide Multiple myeloma C90 LWC LWC LWC DNL
Levodopa / carbidopa Parkinsons G20 DNL DNL LWC DNL
monohydrate
Miglustat Gaucher Disease E75.2 DNL LWC L LWC
Nilotinib Chronic myeloid leukemia C92.1 Ongoing LWC L LWC
Sildenafil citrate Pulmonary arterial hypertension I27 LWC LWC L LWC
Sitaxsentan sodium Pulmonary arterial hypertension I27 DNL LWC L LWC
Carmustine implant Glioblastoma (newly diagnosed) C71 LWC DNL L
poliferprosan
Dasatinib Acute lymphoblastic leukemia C91.0 LWC L DNL
Eculizumab Paroxysmal nocturnal D59.5 DNL DNL DNL
haemoglobinuria
Idursulfase (Iduronate-2- MPS II, Hunter Syndrome E76.1 DNL DNL DNL
sulfatase)

10. Level of agreement between agencies in the HTA recommendations
measured by kappa scores
CDR NICE PBAC TLV SMC
CDR - 0.038 0.165 -0.001 0.062
NICE - 0.178 0.228 0.105
PBAC - -0.023 0.132
TLV - 0.066
SMC -
Source: the authors
Interpretation of kappa score indicators
Poor Fair Moderate Good Very good
0 0.4 0.6 0.8 1
0.2
Source: Altman DG. Practical Statistics for Medical Research. (1991) London England: Chapman and Hall.

11. Associations between HTA bodies and their recommendations
CORA for all drugs (N=287) CORA for 3 therapy areas (N=141)
Legend: Recommendations/outcomes: L: list; LWC: restrict; DNL: reject;
HTA bodies: TLV - Sweden; SMC - Scotland; CDR - Canada; PBAC - Australia; NICE - England & Wales
Therapy areas: c – cancer; n- central nervous system; o- orphan indications
Source: the authors

12. Examples of reasons for differences identified in a number of
case studies (e.g. metastatic colorectal cancer, schizophrenia)
Differences are a consequence of:
• National considerations (e.g. national preferences, costs)
• HTA processes
• Evidence (e.g. clinical expertise)
• Methods (e.g. economic model, choice of comparator)
• Interpretation (e.g. what constitutes acceptable levels of evidence, clinical
endpoint)
• Other considerations (e.g. disease severity, symptomatic/curative, existing
treatment alternatives, orphan indication)

13. Discussion &
Policy implications

14. Inter-country variability in HTA recommendations:
• Low level of agreement between agencies
• Differing associations between:
• HTA bodies and recommendations issued
• HTA bodies and recommendations issued per therapy area
• Differences are a consequence of:
• National considerations (e.g. national preferences)
• HTA processes (e.g. evidence, methods, evidence
interpretation, other considerations)

15. Does it really matter that differences exist across countries?
• Aim of HTA:
• Efficiency in healthcare resource allocation
• Value for money
Therapy X is deemed cost-effective in country A, but not in
country B because of, for example, different levels of evidence
presented (e.g. clinical expertise)
It is individual interpretation of value for money according to
special circumstances
What does all this mean for HTA methodologies?

16. Need to understand why such differences exist: Ways forward
• Identify the reasons for these differences and differentiate
when they are a consequence of:
• National considerations
• HTA processes
• These preliminary findings have demonstrated that it is also
important to differentiate HTA processes per therapy area

17. THANK YOU FOR YOUR ATTENTION