1. School of Health and Related Research (ScHARR) The University of Sheffield Regent Court 30 Regent Street Sheffield, S1 4DA Website: www.sheffield.ac.uk/scharr

3. Portfolio of specialities of HEDS 27/10/11 © The University of Sheffield

4. RESEARCH

5. Information resources Bayesian Statistics in Health Economics Evidence review and synthesis Health Economics and Outcomes Research Cost-effectiveness modelling to support Healthcare Decision Making Main Focus of Research Areas

6. 27/10/11 © The University of Sheffield Consultancy within HEDS Modelling alongside clinical trials Health related quality of life Clinical Trial Simulation Health Economics Modelling Evidence Synthesis Costing studies Systematic reviewing

9. Cost-effectiveness modelling to support Healthcare Decision Making 27/10/11 © The University of Sheffield

12. Examples of cost-effectiveness studies conducted in HEDS 27/10/11 © The University of Sheffield

13. Evaluating the cost-effectiveness of diagnostic tests Dr Matt Stevenson Reader in Health Technology Assessment; NICE Appraisal Committee Member; Contributor to the NICE Diagnostic Assessment Programme manual

15. Cost-effectiveness analyses In the last 10 years there has a been a considerable increase in the importance of cost-effectiveness analyses. This was due to the relatively fixed budget and a combination of ageing populations and emerging expensive interventions. This has led to the formation of funding agencies in England and Wales, Scotland, Australia and Canada.

16. Does a diagnostic test represent value for money? Diagnostic tests with high prices may be cost-effective (i.e. a worthwhile use of a limited budget). Conversely, diagnostic tests with low prices may not be cost-effective. The ‘gold standard’ approach for determining whether the price of a diagnostic test is justified is through an economic evaluation, or cost-effectiveness analysis.

17. Cost-effectiveness analyses The goal of funding agencies is to provide the greatest amount of health for society within the budget, and thus opportunity cost is a key principle. That is, what health would be lost if money was diverted from one intervention in order to fund another. The process is typically to estimate the cost-effectiveness of an intervention through modelling, and comparing this result with a value assumed to represent opportunity cost.

19. Methods for evaluating diagnostic tests There have been, for some time, clear methods guide for undertaking evaluation of pharmaceutical interventions. Recently NICE has set up a Diagnostic Assessment Programme which has issues an interim statement of the methods it expects to be followed in evaluating diagnostics. http://www.nice.org.uk/media/164/3C/DAPInterimMethodsStatementProgramme.pdf

20. Simplified Overview of the modelling required The following slides discuss the steps that would be required to generate an estimate of the cost-effectiveness of a diagnostic test (or series of diagnostic tests). The overview is a simplification. More detailed discussion is provided in the previously listed HTA reports (all free to download) and the Diagnostic Methods statement

21. Estimating Test Accuracy The sensitivity and specificity of a diagnostic test must be estimated. These values would be combined with the estimated prevalence of the condition being tested for, to form an expectation of the number of true positives, true negatives, false positives and false negatives generated by the diagnostic test.

22. Modelling the patient experience For each of the four groups defined, an estimation of the events that would occur to the patient must be modelled. These may differ due to underlying risks and the chosen medical management. The modelling would include factors such as the risks of mortality, risk of morbidity, length of stay within hospital, costs for initial and subsequent care, treatment-related adverse-events and the quality of life for patients in each potential health state.

23. Modelling the patient experience Ultimately, an estimation of the life years, quality adjusted life years (QALYs*) and costs can be attributed to each of the four groups. These can be weighted by the proportions in each group to form a total cost and total QALY for patients post diagnosis. The costs of the diagnostic tests performed are then added. * The QALY is a combination of life years and patient utility. A person living for 10 years at a utility of 0.5 would gain 5 QALYs; a person living for 4 years at a utility of 0.75 would gain 3 QALYs

24. Calculating an ICER* Assume that post diagnosis, an average patient was expected to gain 10 QALYs at a cost of £20,000 under current best practice. These values became 11 QALYs at a cost of £18,000 following a new diagnostic test, which costs £4,000 per patient. In this instance the increase in cost is £2,000 (£18,000 - £20,000 + £4,000) The increase in QALYs is 1. (11 – 10) * An Incremental Cost Effectiveness Ratio.

25. Calculating an ICER In this example, the ICER would be £2,000 per QALY gained (£2,000 / 1) This would be compared with an estimation of the cost of gaining a QALY in interventions that are likely to be replaced.* Thus if this were the result from a real technology appraisal the diagnostic test would be likely to be recommended for use. * NICE has estimated this to be in the region of £20,000-£30,000

26. Implications for diagnostic pricing Where a new diagnostic test has a large impact on mortality or on the utility of a patient, then the QALY gained over the current diagnostic will be greater. ICER = Δ Cost / Δ QALY Thus, for a constant ICER, such a test would be able to command a higher price than a test with a smaller QALY gain.

27. Sequences and subgroups Note that sequences of tests and only incorporating tests on a subgroup of the population are possible. The following slide shows the predicted optimal strategy for diagnosing whether a patient has deep vein thrombosis. The costs of diagnostic tests, the risks of death, morbidity, recurrence, treatment-related adverse-events and the costs of treating future events were all considered in the model.

28. Example of diagnostic algorithm Taken from Goodacre et al. QJM 2006; 99:377–388

29. Returning to the example of sepsis Current gold standard is blood culture, but this has poor sensitivity. A new test is available (SeptiFast © ) which has higher accuracy than blood culture tests, but is relatively expensive. The price may preclude use in all patients with suspected sepsis. A decision support system is also available, (Treat © ) which can categorise patients into high, medium and low risk. This may allow SeptiFast © to be used more efficiently.

31. Estimating the cost-effectiveness of a Treat © and SeptiFast © diagnostic strategy Thus there will be a QALY gain (and cost reduction) associated with the new diagnostic tests. It is expected that these will be greatest in those patients denoted high risk. Factoring in the costs of the diagnostic (Treat © for all patients, SeptiFast © for the groups being evalauted) will allow the cost-effectiveness of diagnostic strategies to be evaluated.

32. Additional complications with evaluating diagnostic tests There are reasons why evaluating diagnostic tests are more difficult than evaluating pharmaceuticals. Due to time restrictions these will be mentioned very briefly under broad headings.

47. An example – screening for CRC 27/10/11 © The University of Sheffield

49. An example – bevacizumab for MCRC 27/10/11 © The University of Sheffield

52. An example – Colorectal Cancer Whole Disease Model

53. From piecewise CPQ to constrained maximisation 27/10/11 © The University of Sheffield

63. CRC natural history model structure 27/10/11 © The University of Sheffield

64. Methods: Modelling solutions 27/10/11 © The University of Sheffield Polyp detection rate at FS screening Polyp prevalence Sensitivity of screening test to polyps

65. Methods: Modelling solutions 27/10/11 © The University of Sheffield Probability of diagnosis of CRC (due to symptoms or chance detection) State allocation diagram CRC incidence in absence of screening

66. Methods: Modelling solutions 27/10/11 © The University of Sheffield Parameter values are estimated by using a Bayesian approach in which sets of parameter values for which the model predictions have a good fit to observed data are generated.*

78. Costs State in the model Perspective NHS & PSS Societal Employer At work £0 £0 £0 1 wk to 6 months sick leave Cost of usual care and intervention incurred by NHS NHS & PSS costs + Employer costs - Transfer costs Cost of intervention incurred by employer + Cost of replacing employee + Production loss over friction period + Salary of replacement employee after friction period + Occupational sick pay + Employer’s NI contribution 6-12 months sick leave Cost of usual care Cost of usual care Occupational sick pay + Employer’s NI contribution 12 months+ sick leave Cost of usual care Cost of usual care

81. Results: NHS/ Societal perspective (1) Cost per QALY gained = £2,758

82. Results: NHS/ Societal perspective (2)

83. Results: Employer perspective Cost per day on sick leave avoided = £0.34

88. Public health modelling: lessons learned from a contraception case study Hazel Squires, Jim Chilcott, Nick Payne, Lindsay Blank, Monica Hernandez, Louise Guillaume ScHARR, University of Sheffield

92. Conceptual model 27/10/11 © The University of Sheffield Education/ employment impacts Impact on ‘unintended’ pregnancies Intervention to encourage contraceptive use Sexually transmitted infection (STI) rate Treatment of STIs Impact upon contraceptive services Abortion Miscarriage/ ectopic Pregnancy/ stilbirth Birth Mistimed Unwanted Low birth weight baby Maternity care Social care Mental health problems (eg. depression) Child health problems Crime Government-funded Benefits

94. 27/10/11 © The University of Sheffield Outcomes of teenage birth Family, societal and individual characteristics Long term outcomes of child Immediate birth outcomes Long term outcomes of parent(s) Low social class Teenage birth Poor education Poor employment More claims for Means-tested Benefits Teenage pregnancy Crime Low birth weight Foetal death Poorly educated mother Poor behaviour/ education as child Other observable or unobservable characteristics A B C

96. Model schematic 27/10/11 © The University of Sheffield t t+1 t+1 t Conception (may be mistimed or unwanted) No conception (may be delayed) Abortion Birth Miscarriage/ ectopic pregnancy/ stillbirth Additional proportion of low birth weight babies Additional Benefit claims of teenage parents Cohort of young people STI No STI t+1 t t t+1

99. Cost-effectiveness decision rules 27/10/11 © The University of Sheffield More expensive Less effective More expensive More effective Less expensive Less effective Less expensive More effective Difference in effectiveness Difference in costs NO YES

100. Results (excl. Benefits) Cohort of 100,000 14-year olds followed over a lifetime 27/10/11 © The University of Sheffield Expected cost per age 14 – 16 pregnancy averted = £37 (condoms) & £110 (hormonal) compared with no intervention Approx. 50% probability that cost-saving

101. Results (incl. Benefits) Cohort of 100,000 14-year olds followed over a lifetime 27/10/11 © The University of Sheffield Dominates no intervention

111. The decision problem To evaluate the cost-effectiveness of high dose statins (atorvastatin 80mg/d, rosuvastatin 40mg/d & simvastatin 80mg/d) versus simvastatin 40mg/d in individuals with acute coronary syndrome.

112. Markov health states

113. Clinical Data Literature review: 28 RCTs > 12 week duration Benefit: LDL-c Synthesis: MTC Relationship: LDL-c & CV events

114. Mean Relative Risks (95% CI) Atorvastatin 80mg/d Rosuvastatin 40mg/d Simvastatin 40mg/d Simvastatin 80mg/d Non-fatal MI 0.425 (0.302 - 0.544) 0.378 (0.241 - 0.51) 0.588 (0.500 - 0.675) 0.510 (0.404 - 0.613) Non fatal stroke 0.623 (0.508 - 0.737) 0.593 (0.465 - 0.717) 0.730 (0.647 - 0.813) 0.679 (0.580 - 0.777) Stroke death 0.809 (0.429 - 1.242) 0.794 (0.384 - 1.261) 0.863 (0.593 - 1.173) 0.837 (0.516 - 1.206) CHD death 0.580 (0.445 - 0.715) 0.545 (0.397 - 0.692) 0.699 (0.601 - 0.796) 0.642 (0.527 - 0.758)

115. Adherence to therapies Scenario A Scenario B Yr 1 Yr 5 Yr 1 Yr 5 S40 ITT ITT ITT ITT A80 ITT ITT 95% 85% R40 ITT ITT 95% 85% S80 ITT ITT 95% 85%

116. Treatment costs Annual cost Sensitivity analyses Atorvastatin 80mg/d £368 £92 Rosuvastatin 40mg/d £387 Simvastatin 40mg/d £17 Simvastatin 80mg/d £34 Monitoring cost £76

118. Results – reduced cost for Atorvastatin simvastatin 40mg/d simvastatin 80mg/d atorvastatin 80mg/d rosuvastatin 40mg/d 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 £0 £5,000 £10,000 £15,000 £20,000 £25,000 £30,000 Threshold ratio Probability cost effective

120. A cost-effectiveness model of prostate cancer screening Matthew Mildred, Jim Chilcott, Silvia Hummel ScHARR, University of Sheffield

128. Screening strategies investigated 05/04/2011 © The University of Sheffield No. Screens Screening Age (years) Screening Interval (years) Single 50 N/A 55 60 65 70 Repeat 50-70 2, 4 50-74 1, 2, 4 55-70 2, 4 55-74 2, 4

130. Definitions & terms used 05/04/2011 © The University of Sheffield PCa Onset Screen Detection PCa Mortality Clinical Diagnosis Other Cause Mortality PCa Onset Screen Detection PCa Mortality Clinical Diagnosis Other Cause Mortality Lead-time Lead-time Over-detection: Relevant:

131. Disease natural history model 05/04/2011 © The University of Sheffield

132. Data 05/04/2011 © The University of Sheffield Data Source Age specific cancer incidence Office of National Statistics Cancer stage distributions ProtecT RCT UK Cancer Registry (ERIC) Gleason score distributions ProtecT RCT UK Cancer Registry (ERIC) PSA/biopsy test characteristics ERSPC RCT (Rotterdam section) Progression Free Survival ERSPC RCT (Rotterdam section) Overall Survival ERSPC RCT (Rotterdam section)

133. Calibration process 05/04/2011 © The University of Sheffield

134. Total SSE during calibration 05/04/2011 © The University of Sheffield

135. Validation: Incidence 05/04/2011 © The University of Sheffield

136. Validation: PCa mortality 05/04/2011 © The University of Sheffield

137. Validation: BAUS 05/04/2011 © The University of Sheffield

138. Results: Incidence 05/04/2011 © The University of Sheffield

139. Results: Mortality 05/04/2011 © The University of Sheffield

140. Over-detection & Lead time: 05/04/2011 © The University of Sheffield Once at 50 50-74 every 4 years 50-74 every 2 years 50-74 every year Over-detection rate 18% 44% 45% 46% Lead time (for over-detected cases) 15.2 yrs 11.6 yrs 12.5 yrs 13.0 yrs

142. Have you heard our findings? 05/04/2011 © The University of Sheffield BBC News 06/12/2010 http://www.bbc.co.uk/news/health-11930979

165. Base case results 27/10/11 © The University of Sheffield

175. Base case results

178. Evidence Review and Synthesis 27/10/11 © The University of Sheffield

182. Examples of Systematic reviews conducted in HEDS 27/10/11 © The University of Sheffield

189. Results 27/10/11 © The University of Sheffield

191. PET results – forest plots

192. PET sensitivity analyses

194. MRI results – forest plot 27/10/11 © The University of Sheffield

199. PET results – ROC plot 27/10/11 © The University of Sheffield All PET studies, showing ROC curve (solid line), mean sensitivity/specificity (black spot) and 95% confidence region (dashed ellipse)

200. MRI results – ROC plot 27/10/11 © The University of Sheffield All MRI studies, showing ROC curve (solid line), mean sensitivity/specificity (black spot) and 95% confidence region (dashed ellipse)

201. Systematic reviews of relevant data Myfanwy Lloyd Jones Senior Research Fellow ScHARR, University of Sheffield Email: m.lloydjones@sheffield.ac.uk

222. Reporting of outcomes: diagnostic intervention (2) 27/10/11 © The University of Sheffield Reference standard positive Reference standard negative Index test positive True positive (TP) False positive (FP) Index test uninterpretable Uninterpretable (U 1 ) Uninterpretable (U 2 ) Index test negative False negative (FN) True negative (TN) Sensitivity = (TP/(TP+U 1 +FN))x100 Specificity = (TN/(TN+U 2 +FP))x100

223. Reporting of outcomes: diagnostic intervention (3) 27/10/11 © The University of Sheffield

228. Systematic reviews of relevant data Myfanwy Lloyd Jones Senior Research Fellow ScHARR, University of Sheffield Email: m.lloydjones@sheffield.ac.uk

240. AUROCs from key studies: test vs liver biopsy Test Degree of fibrosis Study AUROC (95% CI) ELF ‘ Moderate/severe’ Rosenberg 0.94 (0.84-1.00) FibroTest F2-F4 Naveau 0.83 (0.81-0.87) Nguyen-Khac 0.79 (0.69-0.90) Cirrhosis (F4) Naveau 0.95 (0.94-0.96) Nguyen-Khac 0.84 (0.72-0.97) FibroScan Severe fibrosis (F3-F4) Kim 0.98 (0.94-1.02) Mueller 0.91 + 0.03 Nahon 0.94 (90-0.97) Nguyen-Khac 0.90 (0.82-0.97) Cirrhosis Kim 0.97 (0.93-1.01) Mueller 0.92 (0.87-0.97) Nahon 0.87 (0.81-0.93) Nguyen-Khac 0.94 (0.87-0.98)

241. Sensitivity and specificity: ELF Test (subgroup with ALD) Degree of fibrosis Study Threshold score Sensitivity Specificity ‘ Moderate/severe’ Rosenberg 0.087 100% 16.7% 0.431 93.3% 100%

242. Sensitivity and specificity: FibroTest (all patients ALD) Degree of fibrosis Study Threshold score Sensitivity Specificity Moderate-severe (F2-F4) Naveau 0.30 84% 66% 0.70 55% 93% Cirrhosis (F4) Naveau 0.30 100% 50% 0.70 91% 87%

243. Sensitivity and specificity: FibroScan (all patients ALD, all biopsied) Degree of fibrosis Study Threshold score Sensitivity Specificity Moderate-severe (F2-F4) Nguyen-Khac 7.8 kPa 80% 90.5% Severe (F3-F4) Mueller 8.0 kPa 91% 75% Nguyen-Khac 11 kPa 86.7% 80.5% Nahon 11.6 kPa 87% 89% Cirrhosis (F4) Mueller 11.5 kPa 100% 77% 12.5 kPa 96% 80% Nguyen-Khac 12.8 kPa 100% 75.4% 19.5 kPa 85.7% 84.2% Nahon 22.7 kPa 84% 83% Kim 28.5 kPa 90% 87%

244. Sensitivity and specificity: FibroScan (all patients ALD, only subset biopsied) Degree of fibrosis Study Threshold score Sensitivity Specificity Severe (F3-F4) Janssens 13.9 kPa 81% 59% 15.8 kPa 75% 70% 16.5 kPa 72% 70% 17.0 kPa 72% 76.5% 17.3 kPa 69% 76.5% Cirrhosis (F4) Janssens 19.6 kPa 80% 76% 21.1 kPa 75% 80% 23.5 kPa 65% 83%

245. FibroScan: exclusion of patients with laboratory signs of ASH Degree of fibrosis No of patients AUROC Threshold score Sensitivity Specificity F4 101 0.921 11.5 100% 77% 12.5 96% 80% F4 without GOT >100 U/L 86 0.944 11.5 100% 84% 12.5 95% 90% F3-F4 101 0.914 8.0 91% 75% F3-F4 without GOT >100 U/L 80 0.922 8.0 87% 87%

251. Health Economics and Outcomes Research 27/10/11 © The University of Sheffield

255. Examples of health outcome studies conducted in ScHARR 27/10/11 © The University of Sheffield

256. Developing and testing condition-specific preference-based measures: Lessons learnt and policy implications John Brazier, Donna Rowen, Ifigeneia Mavranezouli, Aki Tsuchiya , Tracey Young, Yaling Yang, Michael Barkham