Slides of the speech at G20-Global solutions summit, Berlin 28-29 May 2018. Performance-Based Incentives, Research Evaluation Systems and the Trickle-Down of Bad Science MAY 2018 | PUBLIC & PRIVATE INSTITUTIONS | INSTITUTIONS, POLICY & POLITICS DOWNLOAD PAPER Alberto Baccini’s presentation for INET’s panel on research evaluation at the G20 Global Solutions Summit in Berlin, May 2018. Governments or university managers, vice-chancellors or rectors need information about effectiveness of research funding. Governments are increasingly using monitoring tools as instruments for governance. The promise of bibliometric tools or journal rankings for reducing complexities is a feature that managers, policy-makers and academics find appealing. Systems of incentives are designed for steering scientists at a distance and indicators of scientific activities become more and more targets for scientists. It is the era of the governance of science by indicators. My thesis here is that the adoption of a research policy based on research evaluation systems rewarding performance of scientists could be at the root of many deep problems afflicting contemporary science. And it could be also responsible for the growing difficulties of using the best available science for governmental regulation and policy, in particular for economic policy. Text of the speech available at: https://www.ineteconomics.org/uploads/papers/Baccini-Value-for-money-Berlin-final.pdf

1. Performance-based incentives, research evaluations
systems and the percolation of bad science
Alberto Baccini
(Dep. of Economics and Statistics, University of Siena)
INET ID # INO17-00015 is gratefully acknowledged

4. Research evaluation systems
reinforce the incentives of the
publish or perish environment

6. Research evaluation systems reduce pluralism in science

7. Hindering interdisciplinarity
Suffocation of wider social engagement
Following the herd (Heckman)
Avoiding risk in selecting research topics
Known and documented effect of research evaluation systems

8. Growing perverse incentives in academia
“National assessment procedures, such as the Research
Excellence Framework, incentivize bad practices”

10. Adjusting contents of papers for:
1. Boosting productivity indicators
2. Pumping-up citation indicators

11. Gaming the rules: salami slicing

12. A new documented record in salami slicing
39 papers/10 retracted

13. Gaming the rules: manipulating authorship
For being an author is sufficient
to be more senior than the
people who did the experiment

14. Gaming the rules in Italy?
2010
MATHEMATICS

15. Do senior professors game the rules in Italy?
2010
2010
2010
2010
2010
Mathematics
Logic
Accounting
Management
Professor of geometry
Econometrics
2010
Political economy 2010

16. Pump-up the citations in Italy?

17. A new Italian miracle?

18. The introduction of incentives rewarding metrics of
production and citations, corrupt the social norms
and values that should regulate interactions in
science
Priority in discovery ceases to be the main goal of a
scientist
The new goal is gaining appropriate labels and
performing better on metrics
Summary 1

19. Provisional scientific truths emerge from a battlefield
of conflicting ideas as result of the “public use of
reason”.
In the current system rankings of scientists or
journals guarantee definitive scientific truths
In research evaluation systems, a governmental
ranking guarentees the definitive scientific truth of
science
Summary 2

20. Research evaluation systems put freedom of science
at risk
Governments may drift science in the desired
direction by an appropriate design of research
evaluation
The hegemony of a mainstream economic policy can
be reinforced by stiffening the research evaluation
rules (assign top ranks only to mainstream journals)
Summary 3

21. For having good policy advices we need pluralism
and good science. Both are harmed by research
evaluation systems.
Minimize impact of research evaluation systems on
science. Or in other words: reverse the direction of
the last thirty years of science and research policies.
Dismiss the research evaluation systems. (At least
decouple them from authomatic funding mechanism)
A plea for a new research policy

22. Good policy advices arise from the battle of ideas, not from the battle
for achieving labels or conquering a publication spot in a top journal
- Governments should consider seriously:
• the growing bottom-up pressure in science for replacing
“excellence” with “sound science”
• innovative proposals (e.g. random funding) for funding
research in view of fostering pluralism
• to reward “sound” scientists that provides data, methods,
protocols, software openly for replication and reuse
How to change the PoP environment?

23. Performance-based incentives, research evaluations
systems and the percolation of bad science
Alberto Baccini
(Dep. of Economics and Statistics, University of Siena)
INET ID # INO17-00015 is gratefully acknowledged