Introduction to ArtificiaI Intelligence in Higher Education
Translating research for health policy june 2014
1. Translating Research For Health
Policy: Researchers’ Perceptions And
Use Of Social Media
1. David Grande1,*,
2. Sarah E. Gollust2,
3. Maximilian Pany3,
4. Jane Seymour4,
5. Adeline Goss5,
6. Austin Kilaru6 and
7. Zachary Meisel7
+Author Affiliations
1. 1
David Grande (dgrande@wharton.upenn.edu) is an assistant professor of medicine at the University of
Pennsylvania, in Philadelphia.
2. 2
Sarah E. Gollust is an assistant professor in the Division of Health Policy and Management, School of Public
Health, University of Minnesota, in Minneapolis.
3. 3
Maximilian Pany is an undergraduate student at Swarthmore College, in Pennsylvania.
4. 4
Jane Seymour is a graduate student in public health at the University of Pennsylvania.
5. 5
Adeline Goss is a student at the Perelman School of Medicine, University of Pennsylvania.
6. 6
Austin Kilaru is a student at the Perelman School of Medicine, University of Pennsylvania.
7. 7
Zachary Meisel is an assistant professor of emergency medicine at the University of Pennsylvania.
1. ↵*Corresponding author
Next Section
Abstract
As the United States moves forward with health reform, the communication gap between researchers and
policy makers will need to be narrowed to promote policies informed by evidence. Social media represent an
expanding channel for communication. Academic journals, public health agencies, and health care
organizations are increasingly using social media to communicate health information. For example, the
Centers for Disease Control and Prevention now regularly tweets to 290,000 followers. We conducted a
survey of health policy researchers about using social media and two traditional channels (traditional media
and direct outreach) to disseminate research findings to policy makers. Researchers rated the efficacy of the
three dissemination methods similarly but rated social media lower than the other two in three domains:
researchers’ confidence in their ability to use the method, peers’ respect for its use, and how it is perceived in
academic promotion. Just 14 percent of our participants reported tweeting, and 21 percent reported blogging
about their research or related health policy in the past year. Researchers described social media as being
incompatible with research, of high risk professionally, of uncertain efficacy, and an unfamiliar technology
that they did not know how to use. Researchers will need evidence-based strategies, training, and institutional
resources to use social media to communicate evidence.
Social Media
Research
The translation of research evidence into clinical practice is often slow. Considerable
attention has been paid to delays in this translation process, such as the delay in adopting
2. the use of beta-blockers for acute myocardial infarction after evidence supporting their
use became available.1
The National Research Council recently expressed concern about a
similar issue: whether scientific evidence is being adequately communicated to policy
makers and adopted in public policy.2
The scientific evidence produced by health policy and health services researchers provides
a foundation of knowledge for public policy related to health care and population health.
To maximize the return on public investments in research, the findings from this research
need to be communicated effectively to policy makers and other health care stakeholders.
Moreover, the demand for evidence is likely to grow as the nation embarks on
implementation of the Affordable Care Act and addresses the challenges in health care
related to cost, quality, and access that lie ahead.
Researchers and policy makers confront substantial barriers to effective communication.
For example, the two groups have different incentives, social networks, time frames,
vocabularies, and values.3
⇓–5
Academic researchers work in an environment that values
traditional scholarly productivity, has a long time horizon for knowledge generation, and
is largely based within the social networks of academe.5
In contrast, policy makers require
relevant, timely research that is accessible and delivered by trusted individuals with whom
they have established relationships.6
⇓⇓⇓⇓–11
Policy makers value scientific evidence, but it is
just one item to be considered in a setting full of competing influences.12
Researchers vary in the degree to which they actively disseminate their work, such as
writing policy briefs or working with public relations specialists to generate press
coverage.13
Social media represent an expanding tool that could facilitate communication
of scientific evidence to policy makers.
Technologies such as Twitter have created new communication networks and channels
that can deliver information in targeted and efficient ways.14
For example, a health
legislative aide can follow on Twitter trusted individuals who have strong reputations for
culling and translating policy-relevant findings from research journals. Similarly, bloggers
can translate new research into formats that policy makers might find more accessible
than traditional journal articles.15
These tools complement traditional dissemination methods such as issuing press releases,
because they may better target specialized audiences, quickly link to related content (for
example, policy briefs), create opportunities to sustain communication beyond a single
day of press coverage, and facilitate direct relationships with key information
intermediaries or policy makers. Academic journals are increasingly using Twitter, as are
major health agencies, health officials, and health care organizations. As an example,
Twitter’s website indicated 73,000 followers for Health Affairs and 182,000 for the New
England Journal of Medicine as of April 22, 2014; the Centers for Disease Control and
Prevention regularly disseminates public health information to 290,000 followers on
Twitter.
3. Taking advantage of the promise of social media for translation and dissemination will
require engagement by researchers and their institutions. However, little is known about
researchers’ attitudes toward social media and their capacity to use these media.16
In this article we present findings about the attitudes of health services and health policy
researchers toward social media compared to more traditional communication channels.
We also measured researchers’ use of social media.
Previous SectionNext Section
Study Data And Methods
Study Population And Recruitment
We conducted a mixed-methods survey that included an experimental component. The survey was
administered in person in June 2013 as a structured and semistructured interview, containing both closed-
ended and open-ended questions.
Our respondents were 215 health services and health policy researchers recruited from a random sample of
325 people who had preregistered for the 2013 AcademyHealth Annual Research Meeting, the largest
gathering of US health policy researchers. We limited our sample to researchers whose primary affiliation
was with an academic institution.
To invite each of the 325 people to be interviewed at the meeting, we sent them a recruitment letter
introducing the study, along with $2. The letter stated that “the study will examine strategies to improve
knowledge dissemination from academic researchers to policy makers.” We sent follow-up e-mail to schedule
interviews with responders, and we contacted nonresponders up to two additional times by e-mail.
Participants were entered in a raffle for a $200 retail gift card.
Survey Instrument
We designed the instrument to compare researchers’ attitudes toward different methods of disseminating
research and to measure the frequency of the researchers’ dissemination behaviors. We measured attitudes in
the following three ways.
Vignette-Based Randomized Experiment:
We presented each respondent with a vignette in which we described a researcher who published a study with
implications for health policies on childhood obesity in a peer-reviewed journal. We randomly assigned each
participant one of three versions of the vignette, in which the researcher disseminated her findings through
traditional media, social media, or direct outreach to policy makers. The experimental design and vignettes
are described in online Appendix A1.17
We asked participants to rate the activity presented in the vignette across five domains: the efficacy of the
researcher’s mode of dissemination, their confidence about disseminating their own work in that mode, their
peers’ respect for that mode of dissemination, how much the academic promotion process values the mode,
and their overall impression of the researcher described in the vignette. We asked participants to rate each
domain on a scale from 1 to 10, except for overall impression. In that case, we asked participants to rate the
researcher using a feeling thermometer (a rating between 50 and 100 degrees meant that the respondent felt
warm toward the researcher, while a rating between 0 and 50 degrees meant that the respondent felt cool
toward the researcher).18
Rating Of Efficacy Of Dissemination Methods:
On a scale of 1 to 10, we asked each participant to rate the efficacy of eight channels to “communicate
specifically with policy makers and other decision makers.” Two of the eight channels were scholarly
(editorials or commentaries in peer-reviewed journals and original research articles), one was traditional
4. media (articles in newspapers about a research study), two were direct outreach to policy makers (directly
contacting policy makers and issuing policy briefs from an academic center or institute), and three were social
media (Twitter, Facebook, and blogs on journal websites).
In the vignette experiment we provided a contextual narrative that included characteristics of the researcher,
her research publication, and its successful dissemination. However, in this section of the survey, participants
rated the dissemination method without additional context.
Open-Ended Qualitative Questions:
We asked participants how researchers should be using social media, if at all, to disseminate their work.
Frequency Of Respondents’ Dissemination Activities:
In addition, we measured respondents’ reported frequency of dissemination behaviors. We did this by asking
if at any time in the previous year they had engaged in dissemination activities using scholarly channels
(“published an editorial in a journal where you made policy recommendations”), traditional media channels
(“news media covered a study you published,” were “interviewed by a news reporter about an issue related to
your area of expertise”), direct outreach to policy makers (“published a policy brief,” “had direct contact with
a policymaker,” “directly contacted an interest group or community group”), and social media channels
(“blogged, used Twitter, or Facebook to communicate about research or health policy expertise”).
We conducted cognitive pretesting of the survey instrument on a convenience sample of five health services
researchers and adapted the instrument accordingly.
Survey Administration
The in-person structured interviews were conducted in a designated area at the AcademyHealth Annual
Research Meeting. Trained research assistants conducted the interviews and recorded participants’ responses
in writing for the close-ended questions and with a digital audio recorder for the open-ended questions. We
obtained consent prior to all interviews.
Of the 325 researchers in the sample, 215 participated, for a response rate of 66 percent. Twenty-eight
participants were unable to complete the interview during the meeting; instead, they were interviewed by
telephone. The study was approved by the University of Pennsylvania’s Institutional Review Board.
Analysis
We used analysis of variance to compare both mean scores across the three experimental groups—participants
exposed to vignettes that described dissemination through social media, traditional media, or direct outreach
to policy makers—for each of the domains that we measured and mean academic rank. We used multivariate
linear regression models to test for interactions between experimental-group assignment and academic rank
on the key outcomes.
We used chi-square tests of association to test for significant ( ) differences in reported dissemination
behaviors by academic rank. For open-ended questions, we entered deidentified transcripts of responses into
the structured qualitative software QSR NVivo, version 10.0. We used an inductive process to identify the
main themes that emerged in the qualitative text, first comparing emerging themes across a small sample of
text using a consensus approach and then applying the resulting themes to the full sample.19
Limitations
Our research had several limitations. First, we recruited participants from a sample of university health
services and health policy researchers attending the AcademyHealth Annual Research Meeting. The
researchers attending the meeting may not be representative of university researchers overall. In fact, these
researchers’ applied focus and policy interests should bias them toward more, not less, interest in translation
to policy settings. However, our participants were drawn from a random sample (not a convenience sample)
5. of people who had preregistered for the meeting. Therefore, we are relatively confident that the sample is
representative of this population.
Second, we relied on self-reports to measure behavior. This method could be vulnerable to some recall bias as
well as to social desirability bias, depending on the behavior.
Third, as is the case with all survey research, our results could be vulnerable to nonresponse bias. This is true
in spite of our high response rate of 66 percent.
Previous SectionNext Section
Study Results
Exhibit 1 describes the characteristics of the 215 participants. There were no significant differences in
participants’ demographic characteristics across the three experimental groups.
View this table:
In this window
In a new window
Exhibit 1
Survey Participants’ Characteristics, By Experimental Group, 2013
Vignette Ratings
Participants rated two of the dissemination methods—traditional media and direct outreach to policy
makers—favorably across the five domains we measured (Exhibit 2). Dissemination through social media
received neutral scores in the domains of confidence, peer respect, and academic promotion. It also ranked
below the other two methods in the domain of overall impression but was evaluated almost as favorably as the
other two in efficacy.
View this table:
In this window
In a new window
Exhibit 2
Survey Participants’ Attitudes Toward Dissemination Methods, 2013
As a secondary outcome, we tested for differences in the ratings of the three dissemination methods by
academic rank. We found a significant difference by academic rank in how participants rated the efficacy of
the methods, with senior faculty considering social media less favorably than junior faculty (Exhibit 3).
However, we found no significant differences in the other domains (Appendices A2, A3, and A4).17
View larger version:
In this page
In a new window
Download as PowerPoint Slide
Exhibit 3
6. Survey Participants’ Attitudes About The Efficacy Of Dissemination Methods, By Academic Rank, 2013
SOURCE Authors’ analysis of survey data. NOTES Participants in all three experimental groups ranked
efficacy using a scale of 1 (not at all effective) to 10 (very effective). The interaction between academic rank
and efficacy was significant ( ).
Ratings Of Efficacy Of Dissemination Methods
We asked all participants, regardless of their experimental-group assignment, to rate the efficacy of various
dissemination methods. Respondents rated more traditional dissemination methods (direct outreach to policy
makers, traditional media, and policy briefs) favorably and social media methods (Twitter, journal blogs, and
Facebook) unfavorably (Exhibit 4). Higher academic rank was associated with less favorable attitudes toward
Twitter ( ), blogs on journal websites ( ), and newspaper articles ( ).
View larger version:
In this page
In a new window
Download as PowerPoint Slide
Exhibit 4
Survey Participants’ Attitudes About The Efficacy Of Dissemination Methods In Communicating With Policy
Makers And Other Decision Makers, 2013
SOURCE Authors’ analysis of survey data. NOTE Participants ranked the efficacy of each dissemination
channel to communicate research to “policy makers and other decision makers” using a scale of 1 (not at all
effective) to 10 (very effective).
Because the participants had been differentially exposed to a randomly assigned vignette, we tested for
priming effects—that is, whether exposure to different vignettes earlier in the survey (based on random
assignment to an experimental group) influenced the reported attitudes in questions that appeared later in the
survey. We found none.
Reported Dissemination Activities In The Past Year
Sixty-five percent or more of the participants reported using traditional media (being interviewed by a
reporter or having the news media cover their research) or directly contacting a policy maker to disseminate
information in the previous year (Exhibit 5). However, just 14 percent reported tweeting and 21 percent
reported blogging about their research or related health policy.
View larger version:
In this page
In a new window
Download as PowerPoint Slide
Exhibit 5
7. Survey Participants’ Reported Dissemination Activities In The Previous Year, 2013
SOURCE Authors’ analysis of survey data.
Higher academic rank was associated with higher rates of publishing policy briefs ( ), contacting
interest or community groups ( ), having the news media cover a research study ( ), and
being interviewed by a reporter ( ).
Qualitative Results
Using open-ended questions, we explored participants’ views about the role that social media should have in
disseminating academic research. We identified four key themes, each of which is discussed in more detail
below. First, many researchers believe that the culture of social media is frequently at odds with that of
research. Second, researchers believe that using social media entails substantial professional risk. Third,
researchers have low confidence in their ability to use social media. And fourth, researchers are uncertain
about how effective social media are at disseminating research.
Clash Of Cultures:
Participants were critical of two aspects of social media: the media’s technical limitations and the quality of
communication. Participants emphasized that it would be hard to communicate the nuances of research
findings through social media. One participant said, “I would not use something that’s so succinct that it could
allow for misinterpretation.”
Another participant was even more skeptical: “There are variations in findings, and sometimes those
variations and also the precision of the estimates get lost. …There’s a real challenge in doing that, in getting
at the nuances of your study.”
Many respondents were very critical of the perceived subjectivity in the social media environment. One
researcher said, “Anyone could write a 140-word Twitter statement [that] may or may not be true, and there’s
the risk…that you can just get confused with the junk that’s out there.”
Professional Risk:
Many participants had strong beliefs that researchers—and, by extension, their institutions—are critical of
social media. One participant cautioned junior faculty members, saying, “When you’re still under the tenure
gun, having a controversy around you because of something you say on social media could be a huge barrier
in terms of perceptions of you.”
Another participant raised the concern that using social media could look like advocacy and self-promotion,
which is inconsistent with academic norms: “I think there’s a fine line between being a researcher and being
an issue advocate and also similarly [between] being a researcher who wants to get their message out and
being a self-promoter who wants to just have their face everywhere.”
Low Confidence:
Many researchers feared that they would make a mistake on social media and get themselves in trouble. One
participant said: “I have no idea how to do it. I’m afraid to do it because I don’t want to say or do the wrong
thing that would…get me in trouble.”
Others felt unable to keep up with changing social media technology. One researcher commented: “By the
time I understood Facebook, Facebook had become old. Now Twitter is new. By the time I understand
Twitter, it will have become old and something else will come up.”
Some participants concluded that they needed help in using social media to disseminate their research: “I need
to hire [a] spokesperson or a health journalist to help…write the lay dissemination document outside of the
journal publication, that [is] intended to be disseminated in social media.”
8. Uncertainty About Efficacy:
Some participants seemed open to using social media but were not sure if doing so would be effective. One
said, “It seems natural that this is probably the broadest way to disseminate findings, but I don’t know.”
Another said, “I think the field is relatively young, and we need to do a lot more studies around what’s the
evidence that [social media are] effective at policy translation and dissemination.”
We compared perceptions of social media between self-reported users and nonusers of the media. Users were
more likely to believe that social media are efficacious and ought to be used as an adjunct to more traditional
communication channels. These participants described using social media as a way to disseminate a broader
range of studies (even those not published in what one participant referred to as a “top” journal) and to reach a
broader audience (the general public and policy makers), compared to using traditional media.
Users did not question whether social media were effective, but they wanted proven strategies to optimize the
efficacy of their use of the media. They described professional concerns about using social media that were
similar to those of nonusers but often articulated specific strategies to address those concerns, such as using
social media only for professional purposes, exercising extra editorial caution before posting any content, and
following appropriate communication channels within their universities. Social media users often mentioned
wanting their universities, professional organizations, and the journals they write for to provide clearer
standards about appropriate uses of social media.
Previous SectionNext Section
Discussion
As new challenges such as implementation of the Affordable Care Act emerge in public health and health
care, the need to translate evidence into health policy increases. The barriers to disseminating evidence and
incorporating it into policy are high. However, social media are an increasingly common communication tool
that could play an important role.16
In our survey of university-based health services and health policy researchers, we identified three key
findings. First, researchers are concerned about using social media and the media’s compatibility with
academic research. They have concerns about how their peers and institutions perceive social media. Many
researchers also believe that the culture of social media is incompatible with that of science and research,
describing social media as crowded with opinion and “junk.” They were concerned about allowing their
scientific results to be presented in close proximity to opinion pieces and about being perceived as advocates
or as self-promoting.
Second, many researchers were nonetheless optimistic about the efficacy of social media in disseminating
evidence, when we gave them an example of how the media could be used. Some of the ambivalent views that
we observed in the direct ratings of social media and the qualitative results about the efficacy of the media to
communicate scientific findings may result from participants’ lack of familiarity with how the technology
works.
This was evident in responses to open-ended questions. For example, many respondents believed that they
could not communicate anything beyond the 140-character limit on Twitter, despite the widespread practice
of embedding within tweets links to more substantive content. Many participants acknowledged that the
nature of communication is changing and that eventually researchers will need to catch up.
Junior faculty members were more optimistic than their senior colleagues about the efficacy and scope of
social media. This finding may be an effect of age that results from differential knowledge and adoption of
social media technology. Alternatively, senior faculty members may have greater access to direct policy
communication channels because of their stature and reputation.
9. Third, a large majority of researchers are not using social media to communicate research findings. This is in
contrast to very high rates of use of older communication channels such as traditional media or direct contact
with policy makers.
Our findings have two major implications. First, the lack of evidence about social media’s ability to deliver
research findings to decision makers is one of the barriers to the adoption of these media. Participants in our
study wanted evidence about whether they should use the media and how to use them effectively.
Research is needed to clarify whether or not social media are an efficacious way to communicate evidence to
decision makers and, if they are, to identify the optimal strategies to ensure that the evidence presented in
social media will be used by decision makers. The recent interest by the National Institutes of Health20
and the
Patient-Centered Outcomes Research Institute21
in the science of dissemination holds promise for furthering
this work. However, an explicit focus on social media channels should be a priority in this research agenda.
Second, researchers will ultimately need training and technological support and infrastructure to use social
media effectively. Researchers’ academic institutions might be able to provide some of the necessary
resources.
Previous SectionNext Section
Conclusion
Public investments in research on health and health care are substantial. It is essential to maximize the returns
on those investments by making research a key component of the process of developing, implementing, and
refining health policies. Historically, the communication gap between researchers and policy makers has been
large.3
Social media are a new and relatively untested tool, but they have the potential to create new
communication channels between researchers and policy makers that can help narrow that gap. Determining
how health researchers can best use and adapt this new technology to communicate evidence to policy makers
should be a priority for universities, research funders, and scientists.
Previous SectionNext Section
Acknowledgments
The results are to be presented as an abstract at the AcademyHealth Annual Research Meeting, San Diego,
California, June 8, 2014. This research was supported by the Robert Wood Johnson Foundation Health and
Society Scholars Program and the Summer Undergraduate Minority Research Program at the Leonard Davis
Institute of Health Economics, both at the University of Pennsylvania. Sarah Gollust received a research grant
from American International Group. The authors are grateful to Kristin Rosengren, Michael Gluck, Lisa
Simpson, and the AcademyHealth Translation and Dissemination Institute for support and assistance with this
project. The authors thank Joanne Levy, Janet Weiner, and the students in the Summer Undergraduate
Minority Research Program at the Leonard Davis Institute of Health Economics for their assistance with data
collection. The authors also thank Dan Polsky for guidance and input. [Published online June 6, 2014.]
Previous Section
NOTES
1. ↵
1. Krumholz HM,
2. Radford MJ,
3. Wang Y,
4. Chen J,
5. Heiat A,
6. Marciniak TA
10. . National use and effectiveness of beta-blockers for the treatment of elderly patients after acute
myocardial infarction: National Cooperative Cardiovascular Project. JAMA. 1998;280(7):623–9.
CrossRefMedlineSearch Google Scholar
2. ↵
1. Prewitt K,
2. Schwandt TA,
3. Straf ML
, editors. Using science as evidence in public policy. Washington (DC): National Academies
Press; 2012.
Search Google Scholar
3. ↵
1. Brownson RC,
2. Royer C,
3. Ewing R,
4. McBride TD
. Researchers and policymakers: travelers in parallel universes. Am J Prev Med. 2006;30(2):164–72.
CrossRefMedlineSearch Google Scholar
4. ↵
1. Mitton C,
2. Adair CE,
3. McKenzie E,
4. Patten SB,
5. Waye Perry B
. Knowledge transfer and exchange: review and synthesis of the literature. Milbank
Q.2007;85(4):729–68.
CrossRefMedlineSearch Google Scholar
5. ↵
1. Coburn AF
. The role of health services research in developing state health policy. Health Aff
(Millwood). 1998;17(1):139–51.
Abstract
6. ↵
1. Innvaer S,
2. Vist G,
3. Trommald M,
4. Oxman A
. Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res
Policy. 2002;7(4):239–44.
Abstract/FREE Full Text
7. ↵
1. Contandriopoulos D,
2. Lemire M,
11. 3. Denis JL,
4. Tremblay E
. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of
the literature. Milbank Q. 2010;88(4):444–83.
CrossRefMedlineSearch Google Scholar
8. ↵
1. Lavis JN,
2. Robertson D,
3. Woodside JM,
4. McLeod CB,
5. Abelson J
. How can research organizations more effectively transfer research knowledge to decision
makers? Milbank Q. 2003;81(2):221–48.
CrossRefMedlineSearch Google Scholar
9. ↵
1. Lavis JN,
2. Ross SE,
3. Hurley JE,
4. Hohenadel JM,
5. Stoddart GL,
6. Woodward CA,
7. et al.
Examining the role of health services research in public policymaking. Milbank Q. 2002;80(1):125–
54.
CrossRefMedlineSearch Google Scholar
10. ↵
1. Jewell CJ,
2. Bero LA
. “Developing good taste in evidence”: facilitators of and hindrances to evidence-informed health
policymaking in state government. Milbank Q. 2008;86(2):177–208.
CrossRefMedlineSearch Google Scholar
11. ↵
1. Sorian R,
2. Baugh T
. Power of information: closing the gap between research and policy. Health Aff
(Millwood). 2002;21(2):264–73.
Abstract/FREE Full Text
12. ↵
1. Dodson EA,
2. Stamatakis KA,
3. Chalifour S,
4. Haire-Joshu D,
5. McBride T,
12. 6. Brownson RC
. State legislators’ work on public health-related issues: what influences priorities? J Public Health
Manag Pract. 2013;19(1):25–9.
CrossRefMedlineSearch Google Scholar
13. ↵
1. Brownson RC,
2. Jacobs JA,
3. Tabak RG,
4. Hoehner CM,
5. Stamatakis KA
. Designing for dissemination among public health researchers: findings from a national survey in the
United States. Am J Public Health. 2013;103(9):1693–9.
CrossRefSearch Google Scholar
14. ↵
1. Merchant RM,
2. Elmer S,
3. Lurie N
. Integrating social media into emergency-preparedness efforts. N Engl J Med. 2011;365(4):289–91.
CrossRefMedlineSearch Google Scholar
15. ↵
1. Batts SA,
2. Anthis NJ,
3. Smith TC
. Advancing science through conversations: bridging the gap between blogs and the academy. PLoS
Biol.2008;6(9):e240.
CrossRefMedlineSearch Google Scholar
16. ↵
1. Keller B,
2. Labrique A,
3. Jain KM,
4. Pekosz A,
5. Levine O
. Mind the gap: social media engagement by public health researchers. J Med Internet
Res.2014;16(1):e8.
CrossRefSearch Google Scholar
17. ↵To access the Appendix, click on the Appendix link in the box to the right of the article online.
18. ↵
1. Wilcox C,
2. Sigelman L,
3. Cook E
. Some like it hot: individual differences in responses to group feeling thermometers. Public Opin
Q.1989;53(2):246–57.
13. Abstract/FREE Full Text
19. ↵
1. Bradley EH,
2. Curry LA,
3. Devers KJ
. Qualitative data analysis for health services research: developing taxonomy, themes, and
theory. Health Serv Res.2007;42(4):1758–72.
CrossRefMedlineSearch Google Scholar
20. ↵
1. Glasgow RE,
2. Vinson C,
3. Chambers D,
4. Khoury MJ,
5. Kaplan RM,
6. Hunter C
. National Institutes of Health approaches to dissemination and implementation science: current and
future directions. Am J Public Health. 2012;102(7):1274–281.
CrossRefMedlineSearch Google Scholar
21. ↵
1. Selby JV,
2. Beal AC,
3. Frank L
. The Patient-Centered Outcomes Research Institute (PCORI) national priorities for research and
initial research agenda.JAMA. 2012;307(15):1583–4.
CrossRefMedlineSearch Google Scholar