This is research I have published to help improve the extent to which research impacts society. I published this in the Journal of Science Policy and Governance in their special issue on the Future of Science Policy.

1. Ensuring Social Impact at
Every Stage of Technology
Research & Development
Jeremy Pesner
Engineering and Public Policy
June 22, 2021
International Society for Professional Innovation Management

2. 2
How can we make research more impactful?
• US spends > $600B on research annually (source:
OECD)
• Several reasons for this
 Development/improvement of technology
 Greater understanding of the world around us
 Benefit taxpaying citizens
• But how well are we doing on that last one?
• Is science doing all it can to address our problems?

3. 3
Incentives in Academia
 Outputs: Direct results of research (citations, processes, technologies)
 Impacts: Ultimate effects & consequences of outputs on society
 (Godin & Doré, 2005)
Some early-career scientists see gap between their
research & social impact, and often go into
industry (Leeming 2018)
 Hiring & tenure restrict potentially transformative
research (Holbrook 2017)
 There is not always a clear plan to connect research
with any kind of meaningful social impact

4. 4
Research & Impact
• It’s clearly important to many
stakeholders…
• But there is no precise,
universal definition
• Broad frameworks like Public
Values Mapping
 Bozeman & Sarewitz (2011)
 Youtie and Shapira (2017)
 ...but they all rely on qualitative
judgment…
 that may or may not reflect impact
opportunities and the context of lived
experience
“The economic and societal benefits of science investment
are notoriously challenging to evaluate and quantify. It is
often easier to demonstrate the process of generating impact,
and the kinds of impact we achieve, than to measure the
impact itself.”
-Natural Environment Research Council

5. 5
Targeted impact criteria: is it enough?
• Broader Impacts (BIs) have been required in every NSF proposal since 1997
 “The potential of the proposed activity - beyond the research, per se - to benefit the Nation”
(March, 2007)
 Has “professionalized” the BIs – there are some consultants in universities trained in
applying them
• But some dislike them...
 Vague, difficult to evaluation, “passing the buck” on impact (Sarewitz 2011)
 Considered an “add-on” (National Science Foundation 2015)
 Peer reviewers are often unable to evaluate them (Holbrook & Froderman 2011)
 Better to run actual impact-driven NSF program, such as Research Applied to National
Needs (Bozeman & Boardman 2009)

6. 6
So… what to do? (Participatory R&D Evaluation!)
• Bring all the stakeholders together, not just academics and administrators:
 Poor and underserved, Technology transfer offices
 Journalists, Lawyers, VC Invetors
 Artists, local governments, students
 ...and more!
• All have valuable perspectives to share on how to get the most impact to
the most amount of people
 Science isn’t just for scientists and science policymakers to decide
• Builds off concepts of public participation (Rowe and Frewer, 2000) and
Collective Impact framework (Kania and Kramer, 2011)

7. 7
How to Utilize Them?
• Scientific merit evaluated by academics/funding agencies, social impact evaluated
by the multistakeholder group at every step:
 During the research proposal
 Checking in during the research itself
 Collaborating around technology transfer opportunities
 Helping to shape (not just fix) the relevant markets
• This helps scientists!
 Some rail against the “impact agenda” – being told how their research has to be relevant
 But the purpose here is to help researchers find the alignment between their work and its potential value
• This gets everyone on the same page
 No problems with publications vs commercialization vs impact, etc.

8. 8
Conclusion
• Participatory R&D Evaluation aligns stakeholders and
incentives
 Researchers will gain much more specific ideas about broader impacts if they
regularly connect with a broad population
• Ensures all relevant perspectives are heard throughout
 Difficult to conduct world-changing research if everyone has different plans for it
 Science & society have always influenced one another – let’s make that clear in
our processes!
 Jeremy Pesner
jpesner@andrew.cmu.edu