Presented to the 20th International Conference on Electronic Publishing (ELPUB2016), Goettingen, Germany, 8th June 2016 by Tony Ross-Hellauer Abstract: Openness in peer review is no longer a terra incognita. However, there remains a need for further experimentation and careful evaluation of its advantages and disadvantages in practice. OpenAIRE, the European digital infrastructure for Open Scholarship, offers a unique environment for such experiments. This paper describes the design and early results of three such experiments developed in close collaboration with selected publishing and repository communities. OA Conference Proceedings: http://ebooks.iospress.nl/publication/42901

1. Peer Review on the Move
from Closed to Open
Tony Ross-Hellauer
State and University Library,
University of Göttingen
Goettingen, 8th June, #elpub2016
@tonyR_H

2. 2
OpenAIRE
Dec. 2009 – Nov. 2012
OpenAIREplus
Dec. 2011 – Dec. 2014
OpenAIRE2020
Jan. 2015 – Jun. 2018
Now in our third project phase …

3. EC Open Access Mandate
Progression
FP7 (2008)
• 20% programme areas
• Deposit in Repositories
• APC payments during project
• ERC OA Guidelines
Horizon 2020 (2014)
• 100% programme areas
• Deposit in Repositories
• APCs during and after project
• Open Data Pilot (100% from 2017)
3

4. 4
Human
Network
50 Partners from every EU country, and beyond
Data centres, universities, libraries, repositories
Digital
Network
… fosters the social and technical links
that enable Open Science in Europe and beyond.

5. Human Infrastructure
• Local support for Europe’s diverse research landscape
5
Human support network
• 33 expert nodes all over
Europe to helping with:
• OA training and support
• (OA) policy alignment
• Technical assistance
• Outreach to international
community via COAR

6. 6
Literature
Repositories
OA Journals
Funding Info
Validation
Cleaning
De-duplicating
Inferring
Linking
Organiz
-ations
Projects
AuthorsDatasets
Publicat
ions
Data
Providers
…
Monitoring
Reporting
Evaluation
Impact
Classification
Clustering
Analysis
CRIS
systems
A mini EU-CRIS system
Data
Repositories
Metadata
Full text
Usage data
Discovery
Crowdsourcing
Zenodo
APIs
Data Providers OpenAIRE Platform Services

7. Result: Integrated Scientific Information
System
• 13.9 million unique
publications
• 7 million authors
• 690+ data providers
• 202,000 publications linked
to projects from 5 funders
• 5,500 datasets linked to
publications
• 35,000 organizations
7

8. • Support the Universal OA mandate for all projects
• Run post-grant funding pilot for OA publications
• Support the Open Data Pilot
8
+ Research and development into new trends in
scholarly communication
• Linked Open Data
• Data Citation
• Literature-Data Integration
• Legal issues in Open Data
• Metrics for Open Access
• Open Peer Review
New in
OpenAIRE2020 …
Open Peer Review

9. From Open Access to
Open Science
9
Aim: To open up scientific processes and
products from all levels to everyone …
• Open Access (publications, data,
software, educational resources)
• Open Methodology (open notebooks,
study preregistration)
• Citizen Science
• Open Evaluation / Open Peer Review

10. Why Open Peer Review?
• Time
• Accountability
• Bias
• Incentive
• Wasted effort
Picture credit: AJ Cann, CC BY-SA 2.0
Problems with traditional peer review ...

11. Open Peer Review, broadly defined …
11
Traditionally, peer review is ...
• Anonymous: reviewers unknown to authors, or both authors
and reviewers unknown to each other
• Selective: reviewers selected by editors
• Opaque: neither the process nor the reviews are made public
Openness in peer review can refer to ...
• Absence of anonymity (open identity)
• Self-selecting reviewers (open participation)
• Public processes and reviews (open access)

12. Encouraging
experimentation …
• Use OpenAIRE infrastructure to seed
experimentation
• Stakeholder survey (to come)
• Call for Tenders in 2015
• Small grants
• Investigate how OPR might integrate with
OpenAIRE
• Provide case studies for wider evaluation
• Encourage technological experimentation
12

13. • Francophone environmental
sciences journal
• Using the blog platform
hypotheses.org for OPR
• Using hypothes.is for open
commentary
• Treating OPR as a social rather than
a technological problem
13

14. OpenEdition (2)
• Tone: Overwhelmingly cordial and constructive debate, but
authors and referees reported difficulties finding the right
tone
• Ethical question: One author asked for text to be removed
after receiving critical review
• Mediation in Open Commentary: Just because you build
it, does not mean they will come! (Mediation needed to
finding commentators willing to engage)
14

15. • Incentivizing post-publication
peer reviews (with $$$!)
• Capturing reviews from
“journal clubs”
• Platform for reviews of Zenodo
content
• Author survey
• 15

16. (2)

17. (3)

18. (4)

19. Open Peer Review
Module for repositories
• OPR plug-in for (DSpace) repositories to convert them into
functional evaluation platforms
• Includes published reviews, disclosed identities, reviewer
reputation system
• Complete code, with full documentation, available on Github
under an open license:
• https://github.com/arvoConsultores/Open-Peer-Review-Module
19

20. OPRM (2): Implementation
20
Repository of
the Spanish Institute
of Oceanography
Repository of
the Spanish National
Research Council

21. OPRM (3): First researcher reactions
21

22. Future directions:
A call for common standards
• Uncouple peer review from “publishing”
• Repositories are more than pre-/post-print servers!
• Federate OPR services
• We need to agree:
• What OPR is (standardization of vocabulary)
• How we measure its effectiveness (standardization of
experimentation)
• How we describe it for machines (standardization of metadata)
22

23. www.openaire.eu
@openaire_eu
facebook.com/groups/openaire
linkedin.com/groups/OpenAIRE-
3893548
Thank you!
ross-hellauer@sub.uni-goettingen.de
23