Karen Sepucha, PhD, describes what a good decision is, how we measure decision quality and how the decision quality instrument might be used.
This presentation was part of a Shared Decision Making Month webinar -- What Makes a Good Medical Decision? Defining and Implementing Decision Quality Measures.
Sonagachi Call Girls Services 9907093804 @24x7 High Class Babes Here Call Now
Measuring and Improving Decision Quality
1. Measuring and Improving
Decision Quality
Karen Sepucha, PhD
Health Decision Science Center
Massachusetts General Hospital
ksepucha@partners.org
http://www.massgeneral.org/decisionsciences/
2. Disclosure
Dr Sepucha receives research and salary
support from Informed Medical Decisions
Foundation
Dr. Sepucha is on the advisory board for Vital
Decisions, LLC
3. Agenda
What is a good decision?
How to measure “decision quality”?
Knowledge
Matching treatment to goals
How might the survey be used?
4. Case study: Mr. M’s Story
71yo man referred to orthopedics, worsening
right hip pain over past 2 years, x-rays confirm
damage
Orthopedic surgeon’s note: “I went over in some
detail different treatment options. He very much
wishes to proceed with right total hip
replacement.”
Talked with family and friends, saw PCP for pre-
op evaluation
4
6. High quality, patient-centered care
NQF
National Quality Forum
Core Themes:
fully informed
treatments reflect patients’
want, needs and preferences
play a key role in making
healthcare decisions
7. Agenda
What is a good decision?
How to measure “decision quality”?
Knowledge
Matching treatment to goals
How might the survey be used?
8. Measuring Decision Quality
To provide evidence that
- The patient understands key
facts.
-The treatment received is
consistent with the patient’s
personal goals.
-The patient was meaningfully
involved in decision making
Sepucha et al. 2004 Health Affairs; Elwyn BMJ 2006
9. Who made the decision about treatment of
your breast cancer?
“they didn’t say to me, “Well, we could
remove the breast, we could do this,
we could do that.” They just said, “This
is what we’re going to do.” And that
Mainly the doctor
was it—I wasn’t in on the decision.”
“She was compassionate, … [and] gave
me the data that I needed ... We talked
statistics and sizes and measurements
X Both equally
and things that helped me..with my
decision.”
“I made the decision. I’m very happy with
the lumpectomy because that’s what I
wanted to do from the beginning. They
Mainly you [my doctors] didn’t disagree. They didn’t
agree. They just said, “Okay.” They
understood.”
10. Survey development process
ITEM GENERATION
Literature review
Focus groups and
interviews
DRAFT INSTRUMENT
Candidate facts and
goals • Draft items
Patient and provider • Cognitive
importance ratings interviews (~n=5)
(~n=20)
• Medical and literacy FINAL INSTRUMENT
review • Formal evaluation,
• Field testing large, diverse
samples
• Benchmarks and
standards for
reporting
11. Field tests across decisions
Surgical decisions (n=1,221)
Breast cancer surgery (n=237, n=445) and Reconstruction (n=84)
Knee and hip osteoarthritis (n=382; n=127)
Herniated disc (n=183)
Cancer screening (n=338)
Colon cancer screening (n=338)
Medication decisions (n=1,243)
Menopause (n=401)
Depression (n=404)
Breast cancer systemic therapy (n=358)
Underserved populations (n=289)
Colon cancer screening, African American (n=191)
Breast surgery Spanish language, HIspanic (n=98)
12. Measuring
knowledge
Key facts
Mix of gist and
quantitative
13. Knowledge scores – discriminant validity
Usual 58%
DVD 69%
Healthy control 41%
Patients 53%
Providers 77%
0 20 40 60 80 100
Sepucha KR, et al. Spine 2012; Sepucha K et al. BMC Musculoskelet Disord 2011 Jul 5;12(1):149; Lee C, et
al. J Am Coll Surg 2012 Jan;214(1):1-10.
14. Do treatments match patients’ goals?
Key outcome in Cochrane systematic review of
patient decision aids
2009 update: 3 studies reported
2011 update: 13 studies reported
Systematic review of concordance methods
(Sepucha and Ozanne 2010)
Variability in definitions
Variability in calculations
Stacey et al. Cochrane Database of Systematic Reviews. 2011, Issue 10. Art. No.: CD001431; Sepucha K and Ozanne E.
Patient Educ Couns 2010 Jan;78(1):12-23. .
15. Measuring
goals
Achieve or avoid
Discriminate among
options
Challenge of timing
assessment
16. Calculating a match
Logistic regression model (treatment
received) with goals as independent
predictors
Model returns predicted probability of having
surgery based on patients’ goals
Considered “match” if probability ≥0.5 and
had surgery or if <0.5 and didn’t
Source: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMC
Musculoskelet Disord 2011 Jul 5;12(1):149.
17. Validity: How well does model reflect
patients’ preferences?
Treatment preference
Non surgical options 40%
Unsure 59%
Surgery 74%
Treatment Preference
Model predicted probability of surgery
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Model predicted probability, discriminates among those with
different stated treatment preferences, p<0.001 for all comparisons
Source: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMC
Musculoskelet Disord 2011 Jul 5;12(1):149.
18. Do patients get treatments that
match their goals? (n=383)
Had
Had non surgical
Surgery
treatment
Model predicted
Surgery 49% 13%
Model predicted
12% 25%
Non surgical
Those who matched had lower regret and more confidence
Source: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMC
18
Musculoskelet Disord 2011 Jul 5;12(1):149.
19. Is there a “Decision Quality” score?
Composite score
Requires benchmark for considering patients
“informed” (mean of group that watched
decision aid)
Variable across topics, populations
Risk adjustment (e.g. literacy)
20. Agenda
What is a good decision?
How to measure “decision quality”?
Knowledge
Matching treatment to goals
How might the survey be used?
21. What’s the purpose of measurement?
Research Clinical practice Accountability
BasicTranslClinical Care is implemented in Performance
various settings measured and
compared
Detailed Actionable Benchmarks
Theory Feasible Cost/Feasible
Controlled Acceptable Risk adjustment
22. Mr. M’s story, continued
2 years later, pain worsened and night time
pain came back
Went back to surgeon and had replacement
surgery
Good relief of pain, good function, no regrets
23. Summary
Decision quality definition: extent to which patients are informed
and receive treatments match their goals
Well tested survey instruments exist for common topics
Potential uses span research, clinical care, accountability
Research: evaluate different decision support protocols
Diagnostic screen: identify knowledge gaps and goals in advance
of visit
Accountability: documentation required to proceed with elective
surgery
Notas do Editor
We laid out the proposal for for measuring decision quality in a Health affairs article a few years ago. Main concerns were response that focus on guidelines or “setting right rate” that ignored warranted sources fo variation. Instead we wanted to figure out whether the right treatment is being matched with the right patient, to do that you need treatment rates alone are not enough
Three main phases to measure development First item generation – for us that means identifying the key facts and values that are salient for the decisions. TO do this we review clinical evidence for situation, review literature on decision making experiences, run focus groups and patients and providers to learn about their experiences, distill set of candidate facts and values, Those are then rated by samples of patients and providers for importance to select those facts and values that will be included in draft questionnaire. Items in the draft are run through cognitive testing to make sure patients comprehend the questions and responses and that their answers are refelcting what we hope to learn. In addition field testing of the instrument at this phase can help provide some evidence of acceptability (repsonse rates, missing items) test retest and some preliminary validity. Further refinements and formal testing with large diverse samples, result in a final instrument that is ready for widespread use. To date we are in the second phase.