2012 Summer Medical Editor's Meeting: Dominick Frosch, PhD

1. Preliminary findings from a randomized
mixed-methods cross-over study to
compare the performance of DVD and
web-based patient decision support
Dominick L. Frosch, PhD
Associate Investigator, PAMFRI
Associate Professor of Medicine, UCLA
July 31, 2012

2. Knowledge gaps in decision support
n The internet is increasingly becoming the primary
medium for providing patients decision support.
n Few data are available to guide optimal design of
internet decision support and ensure comparable
effectiveness compared to video.
n A handful of studies have examined the efficacy
of internet decision support, but only one study
(2003) directly compared internet and video
decision support.

3. Review of Video and Internet Presentation
n 98% of participants assigned to the Video
group viewed the tape.
n 54% of those assigned to Internet viewed the
entire presentation of 47 slides. 40% never
reviewed any part of the presentation.
Frosch, Kaplan & Felitti, JGIM, 2003.

4. PSA Knowledge
5
4.5
4
3.5
3
Video
2.5 Internet
2
1.5
1
0.5
0
Pre Post
p<.001

5. PSA Knowledge:
Video & Internet Completers
5
4.5
4
3.5
3
Video
2.5 I-Net Comp.
2
1.5
1
0.5
0
Pre Post

6. Research questions
n How do internet and video decision support
compare in terms of:
– Increases in patient knowledge
– User acceptability
– Decision quality
n What proportion of web-based decision support
do individuals review and how much time do they
spend with it?

7. Study design
Figure
1.
Research
design
Primary care
patients (N=90)
Complete baseline quantitative survey - 1
Review Review RA probes
DVD content
Health
selection
Crossroads
Complete quantitative survey - 2
RA probes Review Review
content Health DVD
selection
Crossroads
Complete quantitative survey - 3

8. Study measures
n Decision/disease specific knowledge
n Quantitative user ratings of decision support
n Decision quality
n Role preferences
n Click-stream (para) data on HC use
n Think-out loud while using HC
n In-depth interviews following each review of
decision support

9. Study setting and sampling
n Palo Alto Medical Foundation in Silicon Valley,
CA.
n Participants are active primary care patients at
PAMF who have/are:
– Diabetes
– Benign Prostatic Hyperplasia
– Overdue to consider colon cancer screening
n Participants are randomly selected from primary
care panels and invited by mail/phone.

10. Participant demographics
N=30
Age 64.6 (SD=12.9)
Gender 63.3% Male
Marital status 76.7% married or living with partner
Ethnicity 83.3% Caucasian
10.0% Asian
6.7% Latino
Education 80.0% College graduate or more
Household income 85.7% >$50,000
Health literacy score (range 0-8) 7.9 (SD=.35)

11. How much do patients view on the
Health Crossroads?
DVD first WWW first
70.00% 66.00%
60.00%
54.00%
50.00% 50.00% 51.00%
50.00% 46.00%
45.00%
40.00% 36.00%
30.00%
20.00%
10.00%
0.00%
All BPH CRC Diabetes

12. How much time do patients spend on
the Health Crossroads?
DVD first WWW first Video length
50.0
45
45.0
40.0
35.0 33 32.2
31
30.0
26.7 27.4
25.2 24
25.0
21.0
20.0 17.6 17.9
16.6
15.0
10.0
5.0
0.0
All BPH CRC Diabetes

13. How many videos in the Health
Crossroads do patients click on?
DVD first WWW first
6.0
5.0
5.0
4.0
3.0
2.5
2.0 1.8 1.8
1.5 1.4 1.4
1.0 0.8
0.0
All BPH (25) CRC (27) Diabetes (19)

14. Time spent watching videos in the
Health Crossroads
DVD first WWW first
3.5
3.1
3.0
2.6
2.5 2.4
2.0
2.0
1.6 1.6
1.5 1.4
1.3
1.0
0.5
0.0
All BPH CRC Diabetes

15. Reflections on using Health Crossroads
“I didn’t even look at the videos but just the presence of the videos seemed like it was
getting in my way, it was just filling up space, and you know I didn’t come looking for a
video…these days not knowing what’s going to happen when you click on a video, if you’re
going to get an advertisement, and uh, and then you know, something you know a video,
will take 10 minutes to say something that you can read in a minute.” MALE, 54, CRC
"My reaction is to take the DVD only because everybody likes to watch a DVD, but the
website is better. But see I was forced to read the website. If I had the DVD, I will actually
put the DVD in and play it. I would go to [the website]; I don't know if I would do as intensive
reading as I did here, um, um. There has to be a catch but I don't know what the catch
is...There has to be follow-up. If there's no follow up, then they might just flip through [the
website] and jot down and see a picture and read a line, but not really read the whole thing."
MALE, 61, DIABETES

16. Changes in knowledge - Combined
DVD first WWW first
90.0%
82.0%
79.0% 77.0%
80.0%
70.0%
68.0%
65.0%
60.0% 56.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
Baseline First review Second review
All

17. Changes in knowledge - BPH
DVD first WWW first
80.0%
73.0%
70.0%
63.0%
60.0%
60.0%
48.0%
50.0%
43.0%
40.0%
40.0%
30.0%
20.0%
10.0%
0.0%
Baseline First review Second review
BPH

18. Changes in knowledge – CRC Screening
DVD first WWW first
120.0%
95.0% 98.0%
100.0% 92.0%
78.0% 80.0%
80.0%
60.0% 55.0%
40.0%
20.0%
0.0%
Baseline First review Second review
CRC

19. Changes in knowledge – Diabetes
DVD first WWW first
82.0%
80.0% 80.0% 80.0%
80.0%
78.0%
78.0%
76.0%
74.0% 73.0%
72.0% 71.0%
70.0%
68.0%
66.0%
Baseline First review Second review
Diabetes

20. Understanding how patients use HC
“It wasn’t too busy, you know um, I didn’t get bored, I could easily find something you know,
some websites are too much you know, this was simple here’s all the different tests you can
take and these are all the procedures, I liked the chart you know the summary thing cause
you forget and with the summary then you don’t have to click from page to page cause you
have it all right there so that’s probably what I liked best was the summary chart.” FEMALE,
52, CRC
"I liked [the website] better because you could read something if weren't sure about it you
can go back and look at it." "In any given amount of time, you can absorb just so much
information. So the DVD is a little too fast, but that can't be helped. So the website is a very
good supplement." MALE, 85, BPH

21. Predictors of Health Crossroads
knowledge gain
Variable R2 Change Beta
Age .10 -.17
Education .21 .28
Proportion of pages .05 .18
viewed
Number of videos clicked .10 .40
Cumulative R2 = .46

22. Which medium do patients prefer?
DVD first WWW first
70.0%
64.3%
60.0%
50.0%
43.8%
40.0% 37.5%
30.0%
21.4%
18.7%
20.0%
14.3%
10.0%
0.0%
Prefer video Equal ratings Prefer Health Crossroads

23. Preferences for Health Crossroads
"Maybe I shouldn't say I would never use the DVD. If [my doctor] gave me the DVD as a
package and said view the DVD, you know, and then go to the website, you know, I would
do that. As I was watching [the DVD], I was thinking that I would never go back to this as a
tool. I would watch it once...and be done with it." MALE, 66, DIABETES
"I am not an internet person. I don't sit there and look at this stuff all day long, but I can read
it. I can go back to it. I can access it immediately with a mouse, and get what I want. [I] don't
have to watch the whole thing again to figure out where in the picture it was and I have to go
find this out. So I like that first website very much." MALE, 85, DIABETES
“The DVD you didn’t have any choice you just had to watch whatever came up but this one
you could choose what you wanted to see.” FEMALE, 64, DIABETES
"Everybody sits at a computer and it's just so easy you know, but you get a DVD in mail and
go "Oh I want to watch that" and you throw it on your desk, but you never do it. But
everyone sits at a computer and it's just "click" and we're there." FEMALE, 52, CRC

24. Conclusions
n Data appear to confirm that DVD may be the
superior option, for ensuring that patients are
exposed to all necessary information.
n Knowledge gains appear to be greater for DVD.
n More viewing of videos embedded in HC appears to
drive greater knowledge gains.
n Some suggestions that having both mediums is
valuable for patients – start with DVD, but use HC to
reinforce knowledge gains.

25. Limitations and next steps
n Findings are preliminary, based on 1/3 of ultimate
sample of 90 participants.
n Study is exploratory, not designed or powered for
null-hypothesis significance testing.
n Non-naturalistic laboratory study, but qualitative
data give us insight into how people might use
decision support at home.
n Participant recruitment is currently 85% complete.
Analysis will be complete by year-end.

26. Questions and suggestions?