Presentation at the 2015 annual meeting of the AAAS on survey and qualitative research related to communication goals for scientists.

1. 1
Engagement with intent? Scientists’ views
of communication and why it matters
Anthony Dudo, Ph.D.
Assistant Professor
Stan Richards School of Advertising &
PR
Texas at Austin
John C. Besley, Ph.D.
Associate Professor & Ellis N. Brandt
Chair
Dept. of Advertising & PR
Michigan State

2. 2
brief context
highlight some results from one of our pilot surveys
highlight some results from our qualitative interviews
1
2
3
Talk Overview
objective: share results from an ongoing research
program focused on scientists’ goals for public engagement

3. Scientist communicators: a timely issue
3
• More PES training
• More scholarship
• Pedagogical shifts
• Scientist-to-scientist advice
• Popular books
• Third-party resources
• Active blogging community

4. How often do scientists engage with the public?
4
• 75% have engaged
• 56% in last year
• 74% in last year
• 70% within last 3 years
• 63% with media within last 5
years
• 80% with public within last 5
years
… more than we commonly assume
Source: Peters et al. (2008); Dudo et al. (2014); Poliakoff & Webb (2007); Dunwoody & Ryan (1985); Corrado et al. (2000); Royal Society (2006).

5. Factors commonly linked with engagement
5
derived from quantitative analyses of survey data
attitude
scientists’ personal attitude toward engagement;
their level of derived or perceived enjoyment
skill
scientists’ real or perceived ability to effectively
communicate with diverse audiences
positive
norms
scientists’ sense that their efforts help society
and are supported / encouraged by their peers
media
orientation
scientists who use new media technology and
who are aware of the role publicity can play in
scientific success
1
2
3
4

6. 6
This research …
has provided a strong baseline understanding
of scientists’ activity and perceptions related
to outreach

7. 7
When a scientist engages …
what is she or he hoping to accomplish?
what are their communication goals?

8. The conceptual context of our work
8
It’s time to focus on the outcomes of engagement
Step 1. Understand scientists’ level of public
communication activity.
Step 2. Understand what factors drive scientists to
be active public communicators.
Step 3. Understand what short-term communication
goals scientists seek to achieve (e.g., inform, excite,
build trust).
Step 4. Understand how scientists’ short-term
communication goals can contribute to achieving long-
term desired outcomes (e.g., secure funding, more kids in
STEM).
Broad
Outcomes
Communication
Goals
Activity
Drivers of Activity

9. 9
What goals do scientists prioritize for public communication?
What factors shape scientists’ goals for public
communication?
1
2
Our two key research questions

10. 5 communication goals for scientists
10
Educate
Defend
science
Excite Build trust Set context
Strategic goalsTraditional goals

11. 2013 | AAAS Scientist Survey
11
Sample
• U.S.-based, university-
based Ph.D.s who
were AAAS members
Method
Distribution
• Online (Qualtrics), Tailored Design Method
• All requests sent from AAAS Membership Dept. (to protect privacy)
• Incentive: 1/200 chance to win $500 amazon.com gift card or donation to AAAS
Response Rate
• 390/5,000 = 8% (not adjusted for undeliverable emails)

12. 12
Descriptive Results

13. 2013 Scientist Survey: Goals
4.96
5.34
4.59
5
5.22
4.76
5.59
5.88
5.72
6.04
5.96
5.79
6.14
1 2 3 4 5 6 7
messaging goal average (r = .54)
describing … in ways that make them relevant
framing research … {to} resonate …
trust goals average (r = .54)
demonstrating … openness & transparency
hearing what others think …
getting people excited about science
knowledge goals average (r = .41)
ensuring that scientists … are part of …
ensuring that people are informed …
defensive goals average (r = .63)
defending science …
correcting scientific misinformation
Strategic
goals
13
“How much should each of the following be a priority for online public
engagement?”
All questions had a range of 1-7 where 1 was the “lowest priority” and 7 was the “highest priority”

14. 2013 Scientist Survey: Goals
5.96 5.88
5.59
5 4.96
5.72
5.35
5.18
4.18
4.44
1
2
3
4
5
6
7
defensive knowledge excite trust context
Own priorities
Colleagues' priorities
14
“How big a priority … for your colleagues?”
All questions had a range of 1-7 where 1 was the “lowest priority” and 7 was the “highest priority”
Respondents believe
they
value all communication
goals more than their
colleagues; biggest
discrepancy is
associated
with “trust”

15. 15
Multivariate Results

16. Predictors of each of the five online comm goals
16
Model specification and results via hierarchical regressions
Controls age, gender, ideology, productivity, career level, science
news online / offline, engagement experience
Scientific field biomedicine, chemistry, physics/astronomy, social
science
Attitudes
fairness: external procedural, fairness: external
distributive, personal enjoyment, goal ethicality (goal-
specific)
Efficacy communication training, external efficacy (goal-specific),
internal efficacy (goal-specific)
Norms subjective norms, descriptive norms, perceptions of
colleagues’ communication priorities (goal-specific)
DVs defend | educate | excite | build trust | context
(36%) (37%) (37%) (33%) (31%)*
1
2
3
4
5
* Adjusted R2

17. Key findings from 2013 pilot survey
17
Scientists prioritize online public communication that is designed to defend
science and educate
Scientists find the least value in the goals that are most likely to lead to
positive engagement outcomes: building trust and setting context (i.e.,
framing)
Scientists’ valuations of specific communication goals are associated with key
predictors from the Theory of Planned Behavior (attitudes, norms, efficacy)

18. 2014 | Interviews with Communication Trainers
18
Sample
• Professional science
communication trainers
• N = 25
• Semi-structured phone
interviews
• ~40 minutes long
• Avg. age, 44
• Gender: F 64% | M 36%
Method
45
36
Doctoral degree
Master's degree
Bachelor's degree
Other

19. “To what extent do you think scientists think about their
audiences before they communicate?”
19
… not much, if at all.
“It’s a new idea for almost all of them to have to
really think about their audience.”
“They don’t think too much about the personal
qualities that people bring and their personal
interests.”
“I think it’s directly related to how much prior
outreach they’ve done.”
1
2
3

20. “How much emphasis do you put on working with
scientists to set goals for their communication efforts?”
20
… not much, if at all.
“Other than identifying who their audience is, our
basic communication training doesn’t really talk
about goals.”
“I don’t know that’s something we really focus on.”
“We don’t talk that much about how to choose
them [goals], just that they are something that you
must have and must choose.”
1
2
3

21. (Probe) “What goals do you emphasize and why?”
21
• “how to articulate a clear
message”
• “how to analyze an audience”
• “helping scientists gain
comfort in an outreach setting”
• “promoting message brevity”
(i.e., elevator speech)
of the trainers who said they emphasize goals, most mentioned
…
Are these
communication
skills rather than
communication
goals?

22. (Probe) “What goals do you emphasize and why?”
• “increase people’s sense of
wonder about science”
• “inspire kids”
• “change policy”
• “help people make better
decisions about their personal
lives”
but a handful of trainers mentioned …
(To us) these are
communication
goals; they’re
possible outcomes
a communicator
could seek
These might be
called broad
outcomes from
communication

23. Other topics we discussed during the interviews:
23
analyses are currently underway
Motivation
Specific goals
Evaluation
Development
How comfortable do you think scientists are with 5
specific goals (e.g., informing, exciting, building trust,
etc.)
How do you evaluate your training program? How do you
help scientists evaluate their outreach efforts?
What kind of information do you seek to help you as a
science communication trainer?
How open do you think scientists are to using
communication research to help their outreach efforts?
Research
Why do scientists seek training? Why do they engage?

24. 3 take home messages
24
Our pilot survey suggests that scientists prioritize
traditional communication goals of educating and
defending science
Our qualitative interviews with science communication
trainers suggest more emphasis is placed on
communication skills than helping scientists set & reach
communication goals
Scientists, engagement researchers, and science
communication trainers should be thinking more
strategically about the desired outcomes of engagement
efforts.
1
2
3

25. What’s next?
25
Current project
‣ 2-year NSF-AISL project that is enabling:
‣ Qualitative interviews with engagement trainers
‣ Surveys with members from >10 scientific
societies
‣ Experiments testing messages related to
communication goals
This material is based upon work supported by the
National Science Foundation (Grant No. AISL-
1421214-421723). Any opinions, findings, and
conclusions or recommendations expressed in this
material are those of the authors and do not
necessarily reflect the views of the National Science
Foundation.

26. 26
Please help us …
what communication goals do you think
scientists have for their public engagement?

27. 27
Anthony Dudo
dudo@utexas.e
du
John C. Besley
jbesley@msu.edu

28. Practical Applications
28
Increase perceived ethicality of specific communication goals
Change beliefs about colleagues’ priorities for online communication
Increase perceived impacts of specific communication goals
Increase perceived personal skills related to specific communication goals

29. Regression
results
29
Models yield
strong
explanatory
power
Standardized betas
* p<.05, ** p<.01, ***p<.001

30. Regression
results
30
Not much
going on with
demographics
or field
Standardized betas
* p<.05, ** p<.01, ***p<.001

31. Regression
results
31
Predictors
that cut
across the
goals
Standardized betas
* p<.05, ** p<.01, ***p<.001

32. Regression
results
32
The education
goal is the
most different
in terms of its
predictors
Standardized betas
* p<.05, ** p<.01, ***p<.001

33. Predictors of each of the 5 online comm goals
35
Model specification for hierarchical regressions
DVs
Controls
Norms
Attitudes
Scientific field
Efficacy
age, gender, ideology, productivity,
career level, science news online /
offline, engagement experience
biomedicine, chemistry, physics/astronomy,
social science
fairness: external procedural, fairness: external
distributive, personal enjoyment, goal ethicality (goal-
specific)
communication training, external efficacy (goal-specific),
internal efficacy (goal-specific)
subjective norms, descriptive norms, perceptions of
colleagues’ communication priorities (goal-specific)
defend | educate | excite | build trust | messaging
(36%) (37%) (37%) (33%) (31%)** Adjusted R2

34. Key findings from 2013 pilot survey
36
Scientists prioritize online public communication that is designed to defend
science and educate
Scientists find the least value in the goals that are most likely to lead to
positive engagement outcomes: building trust and tailoring messages
Scientists’ valuations of specific communication goals are associated with key
predictors from the TPB (attitudes, norms, efficacy)
The traditional goal of educating the public turns on a somewhat different set
of factors than the other goals