Introduction to the peer review workshop for the PhD students of the Wageningen Graduate Schools. The goal is to explain peer review, entice PhD students to take part in the peer review process and give some tips on how to start with peer review.
4. What is peer review?
Peer review is both a set of mechanisms and a
principle at the heart of the system for evaluating and
assuring the quality of research before and after it is
funded or published. It involves subjecting research
proposals and draft presentations, papers and other
publications to critical evaluation by independent
experts (peers). The reviewers are usually appointed
by the funding body or the editors
of a journal or other formal channel for
communication to which the work has been
submitted.
Source: RIN (2011). Peer review: a guide for researchers.
www.rin.ac.uk/peer-review-guide.
5. When is peer review employed?
the evaluation of applications for funding, to determine which
applications are successful
the review of reports submitted by researchers once their
funding award has come to an end, to assess whether a
project has been completed satisfactorily
the evaluation of draft conference presentations, journal
articles and monographs, before they are published, to assess
whether they meet quality standards
the evaluation of publications once they have been published,
through reviews and review articles
the evaluation of the quality of work produced by individuals,
teams, departments and institutions to help determine
appointments, promotions and levels of funding.
8. Which editor?
Roles of 113 WageningenUR researchers with 71 Elsevier
journals
● Editor in chief
● Editor
● Reviews editor
● Associate editor
● Editorial board
● Section editor
● Topic editor
● Editorial advisory board
● Book review editor
9. Einstein to the editor of Physical Review
Kennefick, D. (2005). Einstein Versus the Physical Review. Physicstoday, 58(9): 43
http://physicstoday.org/journals/doc/PHTOAD-ft/vol_58/iss_9/43_1.shtml
10. Forms of peer review
Editorial peer review: The editors themselves judge weather a submission
is accepted for publication
Single-blind review: the identities of those who have submitted the
proposal or draft publication are revealed to the reviewers, but not vice versa
Double-blind review: the identities of the reviewers and those whose
submission is being reviewed are hidden from each other.
Open peer review: this term is used to cover at least three different kinds of
arrangement with increasing levels of transparency (eg. HESS):
● the identities of reviewers and submitters are revealed to each other
● the signed reviews themselves are passed in full to the applicants, and
● authors’ draft publications are made available on websites and reviews
and comments are invited from anyone who wishes to do so.
11. Effectiveness of different types of peer review
Types of peer review % agree
(n=4037)
Peer review could be replaced by usage
statistics
15%
Supplementing peer review with post
publication review
47%
Open and published peer review 25%
Open peer review 20%
Double blind peer review 76%
Single blind peer review 45%
Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing
world: An international study measuring the attitudes of researchers.
JASIST, 64(1), 132-161. http://dx.doi.org/10.1002/asi.22798
12. Transparency of the review proces
Do the journals encourage suggestions for reviewers:
http://www.plosone.org/article/info:doi/10.1371/journal.pone.0013345
Clear dates of submission revised accepted published
http://link.springer.com/article/10.1007/s10750-012-1234-6/fulltext.html
Indication of the handeling editor http://link.springer.com/article/10.1007/s10750-
012-1234-6
Thanking the peer reviewers once a year
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0121093
Open Peer Review: http://bmjopen.bmj.com/content/3/9/e003272
13. Predatory journals
They don't look all like Antarctica Journal of Mathematics
Fake editorial boards (are they credible scientists?)
Very quick/consistent period from submission to
acceptance (no date for revision!)
No language editing/poor English
Quality of the articles
Beall's list of predatory publishers
http://scholarlyoa.com/publishers
14. Recent developments
Post publication peer review
PubPeer https://pubpeer.com/publications/20129177
F1000 Prime http://f1000.com/prime/recommendations
PeerJ, marrying publishing and peer review
Cascading peer review
16. There is still no sign of decline
Larsen, P. & M. von Ins (2010). The rate of growth in scientific publication and the decline in
coverage provided by Science Citation Index. Scientometrics, 84(3): 575-603
http://dx.doi.org/10.1007/s11192-010-0202-z
17. In 2006 about 1,350,000 articles were published in
peer-reviewed journals (WoS)
2,000,000 in 2013 (Scopus)
Drivers
● More scientists, more
publications
● Pressure to publish (in
journals)
● In search for the "least
publishable unit"
18. Other criticism
Not always effective at detecting falsification, fabrication
and plagiarism (FFP)
Brings delay in the research an innovation cycle
Selection of reviewers brings bias
Judgement subjective and inconsistent
Tends toward conservatism and stifles innovation
Disadvantageous to interdisciplinary research
Imposes increasing and unsupportable burdens on
reviewers
19. Retractions are related to journal prestige
Fang, F.C. & A. Casadevall (2011). Retracted science and the retraction index.
Infection and Immunity, http://dx.doi.org/10.1128/iai.05661-11
20. Retractions on the rise
Van Noorden, R. (2011). Science publishing: The trouble with retractions.
Nature, 478: 26-28 http://dx.doi.org/10.1038/478026a
21.
22. Peer review, a plan of action
Nicholas, K. A. & W. S. Gordon (2011). A quick guide to writing a solid peer review.
EOS, Transactions American Geophysical Union, 92(28): 233-234
http://dx.doi.org/10.1029/2011eo280001
23. Considering a reviewing request
Do you have the expertise?
Is there a conflict of interest?
Do you have time?
24. After the review process
You should be informed on the editor's decision
● If the editor makes a decision on the manuscript
counter to the direction you recommended in your
review, you may request an explanation.
You should not reveal to the author or authors after
review that you were a reviewer
Do not make public the contents of the manuscript nor
use any information in the manuscript until it is
published.
25. Why should you take part?
New peer reviewers badly needed
You can benefit from peer review
● Critical reading
● Expressing your opinion
● Improve you own writing
● Expand your professional network
26. Some resources
BMJ peer reviewers: resources
http://www.bmj.com/about-bmj/resources-reviewers
Elsevier's reviewers home
http://www.elsevier.com/wps/find/reviewershome.reviewers
Springer peer review academy
http://www.springer.com/authors/journal+authors/peer-review-academy
Nature peer review debate
http://www.nature.com/nature/peerreview/debate/index.html
British Ecological Society http://www.britishecologicalsociety.org/wp-
content/uploads/Publ_Peer-Review-Booklet.pdf
27. Thank you!
On the Web:
@wowter
wowter.net
This presentation
http://www.slideshare.net/Wowter/introduction-to-peer-review
http://tinyurl.com/7r67fmm