Outcomes Talk 2016

1. Technology Assessment,
Outcomes Research and
Economic Analyses
Deborah A. Baumgarten, M.D., M.P.H.
Emory University Hospital
Department of Radiology

2. Rationale
• Cost of medical care increasing faster than
cost of living
• 5.6% of GDP in 1965; 17.6% 2012; 19.8% 2020
• $27.5 B 1960; $2.71 T 2011; $4.64 T 2020
• $8649 per person in 2011; $13,709 2020
• US spends more than 33 others in Organization
for Economic Co-operation and Development
• But fewer MDs, hospital beds per capita

3. OECD spending on health care
OECD Health Data 2012

4. Health care expenditure as % GDP
OECD Health Data 2012

5. 0
2
4
6
8
10
12
14
16
18
1980 1983 1986 1989 1992 1995 1998 2001 2004 2007 2010 2013
US (17.1%)
FR (11.6%)
SWE (11.5%)
GER (11.2%)
NETH (11.1%)
SWIZ (11.1%)
DEN (11.1%)
NZ (11.0%)
CAN (10.7%)
JAP (10.2%)
NOR (9.4%)
AUS (9.4%)*
UK (8.8%)
Health Care Spending as Percentage of GDP, 1980–2013
Percent
OECD Health Data 2015

6. Rationale
• Workers now paying at least $1,094 more in
premiums annually for coverage than in 2000
• Still waiting to see what ACA does…
• Despite expenditures, can we document
significant improvement in health status?
• Life expectancy
• Perinatal mortality

7. Life expectancy

8. Infant mortality rate

9. Our poor health
• Life expectancy 78.8 vs. 81.2 (2013)
• Infant mortality 6.1/1000 vs. 3.5 (2011)
• 68% adults over 65 had at least 2 chronic
conditions (33% UK, 56% Canada; 2014)
• Obesity rate 35.3% (3.7% Japan, 30.6% NZ; 2014)
• Smokers 13.7 (12.8 Australia, 24.1 France; 2013)

10. US does more of…
United States Rank compared
with OECD
countries
OECD median
MRI units 35.5 per 1,000,000 2nd (Japan 1st) 11.4 per 1,000,000
MRI exams 106.9 per 1000 50.6 per 1000
CT units 43.5 per 1,000,000 3rd (Japan 1st,
Australia 2nd)
17.6 per 1,000,000
CT exams 240 per 1000 136 per 1000
Tonsillectomy 254.4 per 100,000 1st 130 per 100,000
Coronary bypass 79.0 per 100,000 3rd 47.3 per 100,000
Knee replacement 226 per 100,000 1st 121.6 per 100,000
Caesarian section 32.9 per 1000 live
births
6th 26.1 per 1000 live births
OECD Health Data 2012 and 2015

11. Why?
• Current payment structure means MDs get
paid more to do more interventions
• ACA has not had effect yet…
• Fear of litigation
• Patients ask for more—and usually do not
pay out of pocket (insurance covers)

12. Why?
• MDs in US paid more but seen less
• 4 visits/year (OECD average 6.5 visits/year)
• US uses more pharmaceuticals
• 2.2 drugs per adult (1.2 Netherlands)
• Cost more (US=100; Australia=49; Germany=95)
• Procedures cost more
• Bypass 75,345 vs. 42,130 in Australia
• Appendectomy 13, 910 vs 4,995 Netherlands

13. Technology Assessment

14. Bit of background
• Technology assessment hierarchy
• Based on efficacy (benefit under ideal
circumstances)
• Differs from effectiveness (benefit in real
world)
• Differs from efficiency (sufficient value?
adds economic component)
Fryback D, Thornbury J. The efficacy of diagnostic imaging. Med Dec Making 1991;11:88-94

15. Technology assessment hierarchy
• Technical efficacy
• Diagnostic accuracy
• Diagnostic impact
• Therapeutic impact
• Patient outcomes
• Societal outcomes
Fryback D, Thornbury J. The efficacy of diagnostic imaging. Med Dec Making 1991;11:88-94

16. Technical efficacy
• Ability of technology to produce image
• Measured by image characteristics
• Signal to noise, resolution of line pairs, etc.
• Developmental: prototype and phantom,
safety
• Exploratory: possible use, wide range of
patients, situations, conditions

17. Technical efficacy
• Ideas of what falls here?

18. Diagnostic accuracy
• Distinguish between disease, non-disease,
or normal, abnormal
• Establish diagnosis if symptoms
• Screen in asymptomatic patient
• Quantify severity or extent of disease
• Prognostic information (staging)
• Monitor therapy
• Need gold standard; measure of truth

19. Diagnostic accuracy
• Sensitivity, specificity, positive and negative
predictive values
• 2 x 2 table to display comparison to gold
standard or reference
• Area under ROC curve
• Most radiology research falls into this
category

20. Diagnostic accuracy
• Ideas?

21. Diagnostic impact
• Which imaging tests have greatest impact on
clinical decisions?
• Before test need to know
• Differential diagnosis
• Degree of certainty of diagnoses
• Probability of each consideration
• Reassess after test performed
• Best done prospectively

22. Diagnostic impact
• Ideas?

23. Therapeutic impact
• Which imaging tests have greatest impact on
therapeutic decisions?
• Percent procedures avoided due to imaging
• Percent imaging changed plan
• Retrospectively inferred from records
• Prospectively assessed using clinicians’
stated plans

24. Therapeutic impact
• Ideas?

25. Patient outcomes
• Traditionally, morbidity or mortality
• Change in quality adjusted life years (QALY)
• Can be difficult to prove link as imaging
study several steps removed from patient
outcome

26. Societal outcomes
• Cost analyses from societal perspective
• Usually reported as cost per QALY
• Must be below some threshold to be
accepted (? $50,000/QALY)
• Example, head CT to screen for brain mets
in lung cancer patients or only if
symptomatic?
• $70,000/QALY deemed too high to adopt

27. Outcomes research

28. Outcomes research—goals
• Use imaging wisely
• Avoid over-diagnosis and over-treatment
• Risks of treatment in early or mild disease
may outweigh benefits
• Data that will permit maintenance of quality
at reduced cost
• Increasingly important
• Radiology needs to prove worth

29. Outcomes research—getting started
• Choose technology to be evaluated
• Define specific clinical indication
• Define target population
• Choose alternative techniques for
comparison (may include no imaging)
• Define outcomes affected by technology

30. Randomized controlled trial
• Gold standard of outcomes research
• Patients assigned to different groups based
on imaging (or no imaging) to be performed
• Measure appropriate outcomes
• Confounders and biases accounted for in
design

31. RCT—limitations
• Cost
• Size of population
• Power calculation
• Estimate expected differences
• Time to complete
• Technology obsolete by time outcomes
assessed?
• Technology as moving target
• Results may only apply to narrow spectrum of
patients; issue of generalizeability

32. RCT—unnecessary when?
• Anecdotal evidence of benefit dramatic;
observational studies suffice
• Example: head CT in trauma
• New technology:
• Has same or fewer side effects
• Has same or better accuracy
• Is less expensive

33. RCT—necessary when?
• Really don’t know answer
• New technology:
• More expensive
• May result in more morbidity
• Very controversial

34. ACRIN—NLST
• National Lung cancer Screening Trial
• 25,000 patients randomized; low dose CT vs. CXR
• Baseline, years 1 and 2
• Q6 month health status and medical intervention
questionnaire
• Very detailed description of what constitutes
positive screen and what to do
• Outcomes
• Primary: Lung cancer specific mortality
• Secondary: huge list

35. ACRIN—NLST
• Low dose CT more sensitive than
radiography
• Two annual incidence screenings with low-
dose CT resulted in
• Decrease number of advanced-stage
cancers
• Increase number of early-stage lung
cancers
NEJM 2013;369:920-921

36. Example to work through
• CT for appendicitis: improve outcomes?
• Technology:

37. Example to work through
• CT for appendicitis: improve outcomes?
• Technology: MDCT

38. Example to work through
• CT for appendicitis: improve outcomes?
• Technology: MDCT (?oral, ?IV, ?rectal)
• Indication:

39. Example to work through
• CT for appendicitis: improve outcomes?
• Technology: MDCT (?oral, ?IV, ?rectal)
• Indication: RLQ pain, fever, etc.
• Population:

40. Example to work through
• CT for appendicitis: improve outcomes?
• Technology: MDCT (?oral, ?IV, ?rectal)
• Indication: RLQ pain, fever, etc.
• Population: patients presenting to ER
• Alternative:

41. Example to work through
• CT for appendicitis: improve outcomes?
• Technology: MDCT (?oral, ?IV, ?rectal)
• Indication: RLQ pain, fever, etc.
• Population: patients presenting to ER
• Alternative: best clinical judgment plus lab
• Outcomes:

42. Example to work through
• CT for appendicitis: improve outcomes?
• Technology: MDCT (?oral, ?IV, ?rectal)
• Indication: RLQ pain, fever, etc.
• Population: patients presenting to ER
• Alternative: best clinical judgment plus lab
• Outcomes: morbidity, mortality, perforation rate,
length of hospital stay, surgery rate, % normal
appendices removed, cost to hospital, etc.

43. Example to work through
• CT for appendicitis: improve outcomes?
• Technology: MDCT (?oral, ?IV, ?rectal)
• Indication: RLQ pain, fever, etc.
• Population: patients presenting to ER
• Alternative: best clinical judgment plus lab
• Outcomes: morbidity, mortality, perforation rate,
length of hospital stay, surgery rate, % normal
appendices removed, cost to hospital, etc.
• Is this feasible?

44. What if RCT not feasible?
• Modeling: Use of decision analysis software
• Need information from other studies
• Accuracy, sensitivity, specificity
• Course of disease, natural history
• Effectiveness of therapy
• Patient utilities and costs
• Meta-analyses for combining results of
studies

45. Modeling
• Assumptions about test parameters
(sensitivity, specificity) based on literature
• Then vary assumptions
• Assumptions about frequency of outcomes
based on personal experience or literature
(mortality and morbidity rates, etc.)
• Allows variation of these

46. Modeling—literature example
Cervical spine screening with CT vs. plain films
Blackmore CC et al. Radiology 1999;212:117-125

47. CT in appendicitis example
• Retrospective study of 651 patients treated before
use of CT for appendicitis (1992-1995)
• 52% went to surgery; 24% for observation then
surgery; 24% observation to discharge
• Initial surgical group, 81% appendicitis
• Observation to surgery group, 80% appendicitis
• Observation to D/C group, assumed no
appendicitis (no returns to their hospital)
Rhea JT et al. AJR 1997;169:113-118

48. CT in appendicitis example
• Apply sensitivity and specificity of CT from
literature to 100 theoretical patients
• Use morbidity, mortality, perforation, correct
diagnosis rates from study of 651 patients
• Model impact of CT on outcomes
• Decrease in nontherapeutic surgery (13
patients), observation days (1 day less on
average), perforation rate, etc.
• Alternative diagnoses also confirmed
Rhea JT et al. AJR 1997;169:113-118

49. Intermediate outcomes
• Choose levels lower in hierarchy
• Diagnostic impact
• Change in diagnostic possibilities
and confidence
• Before and after test ask referring MD:
• Differential diagnosis
• % certainty for each diagnosis
• Degree of satisfaction with test

50. Intermediate outcomes
• Seattle Low Back Pain Imaging Project (SLIP)
• RCT of plain films vs. MR as initial imaging
• Diagnostic impact
• 15% new diagnosis with plain film
• 30% for MRI (spinal stenosis, HNP, etc)
• Change in probability for most likely
diagnosis for HNP (p<.002)
Jarvik JG et al. Radiology 1997;204:447-454

51. Intermediate outcomes
• Therapeutic impact
• Before test, ask MD for treatment plan
• Observe if plan changes after test
• If temporally related, infer causal
relationship between test and plan change
• SLIP: 38% avoided additional test if plain film
first, 64% if MRI first (p<.001)

52. Intermediate outcomes
• Advantages
• Cheaper and easier than full RCT
(RCT may show intermediate outcomes if
assessed as in SLIP)
• No one denied potentially good test
• Works well for “add-on tests”
• Tests that would not necessarily
replace existing work-up

53. Intermediate outcomes
• Disadvantages
• Stated plans do not necessarily equal
clinical action
• If treatment invasive, difficult, expensive
or risky, easier to hypothetically choose
• RCT still better for “replacement test”

54. Patient outcomes
• Reduced mortality
• Reduced morbidity
• Decrease complications from other tests
• Avoid invasive procedures
• Faster return to normal (or pretest) state

55. Patient outcomes
• Quality of life: ability to undertake
rewarding and enjoyable activities
• How person feels, functions
• Quality adjusted life years (QALYs)
• Patient satisfaction, reassurance
• Physiologic marker improvement
• Improved function

56. Functional status
• General and specific measures
• SF-36 (Short Form 36)
• 36 questions, 8 domains
• General health, pain, physical functioning,
social functioning, mental health, etc.
• Roland score (back pain specific index)
• Sciatic frequency index

57. SLIP follow-up
• RCT ultimately enrolled 380 patients
• Roland score at 12 months did not differ between 2
groups (8.75 plain film vs. 9.34 MRI, p = .53)
• p >.75 for all domains of SF-36
• Patient reassurance measure
• 1 month p=.09
• 3 month p=.08
• 12 month p=.002 (58% plain film vs. 74% MRI)
• Increased reassurance with MRI over time
• 10 in MRI vs. 4 in plain film had surgery (p =.09)
Jarvik JG et al. JAMA 2003;289:2810-2818

58. Outcomes research—limitations
• Tenuous link between test and patient outcome
• Test usually steps removed from outcome
• Best situation: proven link between therapy
and outcome  link diagnosis to therapy to
outcome
• Example: detection of cerebral aneurysms
with MRA  treatment decreases morbidity
and mortality  link MRA to reduced
morbidity/mortality

59. Outcomes research—limitations
• If no proven impact on outcome, may be:
• Diagnostic test (inaccurate, insensitive)
• Referring clinician (inappropriate use of
info; info does not reach)
• Therapy (ineffective or unavailable)
• Patient (lack of compliance)
• System (lack of patient access)

60. Alternative outcome
• Therapeutic value of diagnostic test
• Diagnostic test can have direct impact on
quality of life
• Reassurance important to patients and
referring clinicians
• If consider only positive tests fail to consider
that negative test has value

61. Alternative outcome
• Example: nonspecific chest pain
• 176 patients randomized to receive no
studies or EKG and CPK level
• Test group had less short term disability
(20% versus 46%; P=.001)
• Use of diagnostic test independent predictor
of recovery in logistic regression
• Test group felt care “better than usual”
Sox HC, et al. Ann Intern Med 1981;95:680-5

62. Economic Analyses

63. Economic evaluations
• Must establish viewpoint of analysis
• Society, insurer, provider, patient
• All have different inputs and outcomes;
different costs to consider
• Example: patient may value increased
quality of life at any cost while HMO
may value decreasing overhead costs

64. Costs
• Value of resources
• Charges as estimate
• Medicare/Medicaid reimbursement as
estimate
• Micro-costing techniques—all resources
identified and quantified
• Consider cost of missed abnormalities and
superfluous work-up of false positives

65. Costs—examples
• Medical
• Diagnostic test and interpretation
• Nursing care
• Nonmedical
• Time (travel, tests and treatment)
• Travel
• Support with daily activities
• Lost productivity

66. Types of economic analyses
• Cost identification or cost minimization
• Compares alternative health care
strategies assumed equally effective

67. Cost identification—example
• CT vs. US plus plain film after ESWL
• 25 patients underwent all 3
• Assumed equal accuracy
• Exams timed; direct technical costs calculated
using procedural-based accounting system
• CT 15.3 minutes; US + plain film 37.2 minutes
• CT $36.86; US + plain film $57.60
• Sensitivity analysis on time; within reasonable
range CT always cost less
Remer EM et al. Radiology 1997;204:33-37

68. Types of economic analyses
• Cost benefit
• Costs and benefits assigned dollar values
• Comparison of different technologies
• Limited use in medicine due to difficulty
assigning monetary values to health states
• Cost effectiveness leaves outcomes in
natural, objective units
• # lives saved, # cancers detected, etc.

69. CEA—example
• What gain from sixth stool guaiac?
• Progression from fifth to sixth stool guaiac
costs $47 million/cancer detected
• Can use intermediate outcomes (length of
stay, readmission rates, number of
unnecessary surgeries prevented)
Neuhauser D, Lewicki AM. N Engl J Med 1975; 293

70. Cost utility analysis
• Similar to CEA but uses subjective outcomes that
explicitly incorporate patient preferences
• QALY: basic outcome measure
• Year of life with significant morbidity less desirable
than year with excellent health
• Full health equals 1
• Death equals 0
• Morbidity somewhere between 0 and 1
• Preferred method for economic evaluations

71. Utility values
• Side effects from interferon—.93
• Mild angina—.90
• Prophylactic mastectomy, oophorectomy—.86
• Moderate angina—.70
• Hospitalization for tuberculosis—.60
• Severe angina—.50
• Moderate to severe stroke—.39
• Recurrent stroke—.12

72. Components of economic analysis
• Most with modeling software
• Reference case (defines typical patient)
• Strategies (set of alternative care pathways)
• Time horizon (period to consider risks and
benefits)
• Perspective (viewpoint of decision maker)
• Effectiveness measure (outcome)
• Decision tree allows varying components

73. Decision tree
Singer ME and Applegate KE. Radiology 2001;219:611-620

74. Components of economic analysis
• Probabilities (all events in model have one)
• Estimated from historical data; literature, pilot
study or expert opinion
• Utilities (assign utility to every health
condition in model)
• Costs (identify all relevant)
• Assumptions (due to limitations imposed by
available data; must explicitly state)

75. Economic analysis—example
• Whole body CT screening
• Base-case analysis: from cohort of 500,000
50 year old men
• Sex, age varied in analysis
• Benefits of screening assumed due to earlier
detection of disease, improvement in survival
• 8 conditions included in model: ovarian, pancreatic,
lung, liver, kidney and colon cancer; aortic aneurysm,
coronary artery disease
• Costs of screening, follow-up tests, patient care
Beinfeld, M. T. et al. Radiology 2005;234:415-422

76. Beinfeld, M. T. et al. Radiology 2005;234:415-422
Schematic of decision-analytic model with life expectancy
and cost outcomes used in the cost effectiveness analysis

77. Economic analysis—example
• Compared to routine care
• 6 days of life gained at $2513/patient
• Incremental cost: $151,000/life-year gained
• 90.8% had finding; only 2% had disease
• 32.3% of total cost = work-up of FPs
• With favorable assumptions, one time screening not
cost effective; would add burden to healthcare
system
Beinfeld, M. T. et al. Radiology 2005;234:415-422

78. Economic analysis—limitations
• Not all agree with strategies chosen
• Was comparison to standard of care?
• Were all reasonable options considered?
• Was reference case typical of patient population?
• Fears of bias if study funded by drug company or
equipment manufacturer
• If too early, data scant; if late, decisions made and
model obsolete
• Noncontrast CT for renal colic: accepted standard
without study of cost

79. Conclusions
• Need to be familiar with concepts
• Way to inform health care policy makers, guide
research
• Goal to improve quality in rational manner, decrease
waste and unnecessary expenditure
• Enhance Radiology community’s ability to participate
in health care decision-making locally and nationally
• Preserve our specialty!

80. Thanks!

81. US does more of…
OECD Health Data 2015