Peer review is an integral part of scientific publishing. The mode of peer review has been changing as the scientific community keeps searching for a better way to do peer review. Formal peer review, where journal editors enlist the help of external experts to evaluate a manuscript’s suitability for publication, started around 1940s and gradually became normal practice. Traditionally, peer review is in the single blind mode, and this mode is still in use by a large number of journals. In the single blind mode, the authors’ identities are known to the reviewers, but the reviewers remain anonymous. In this traditional mode, the review reports are not made public, keeping the peer review process behind the scene.
There are potential problems associated with this traditional mode, primarily related to the fairness of the review. Reviewers may be biased because of the authors’ identities (e.g., their gender, nationality, past publication records, or the prestige of their institutions). In addition, since the reviewers don’t need to worry about retaliation from the authors, they may be overly critical in their evaluation.
To protect the authors, some journals have adopted the double blind review mode, where the authors’ names and affiliations are removed from the manuscript copy that is sent to the reviewers. A study finds that in computer science conferences, the double blind review mode does help reduce bias, allowing more contributions from newcomers to the conferences, as compared with the single blind mode 1. However, double blind review hasn’t replaced single blind review, perhaps partly because it is often not difficult for the reviewers to find out the identities of the authors, for example, from a published preprint, from the references cited, or from some information in the manuscript.
Another approach journals take to facilitate objectiveness in peer review is to make the reviewers’ identities open to the authors (open identity mode). Another form of open peer review is to publish peer review report together with the manuscript (open report mode). The open report mode is generally thought to encourage constructive reviews. Further, this adds transparency to the review process and helps to build readers’ trust in scientific research. One concern with open review is that the reviewers may not be as critical as they should be, but a randomized controlled trial found that the review quality did not suffer when reviewers were told that their signed review reports would be published 2. Many publishers now view the benefits of open review to outweigh its disadvantages. Several organizations in life sciences “advocate for open reports as the default and for open identities to be optional, not mandatory” 3, and their open letter pledging to publish peer reviews has so far been signed by 336 journals 4. As this trend is in line with the global movement toward making research results openly accessible, more journals are likely to join.
Some journals are experimenting with crowd review, where a large number of reviewers are involved in peer review of a given manuscript, with each reviewer giving a short comment. For example, Synlett offers Select Crowd Review as a review option to authors, and Atmospheric Chemistry and Physics uses Interactive Public Peer Review to supplement reviews from invited referees. It is probably too early to know if other journals will adopt this mode.
Despite all these different modes, some issues currently associated with peer review still await innovative solutions. One is that there is a significant imbalance in the peer-review effort; a study finds that “20% of researchers perform 69% to 94% of reviews” in biomedical research 5. Another issue is repeated reviews of individual manuscripts when a paper is rejected by a journal and the next journal the paper is submitted to conducts a new round of peer review. The mode of peer review is still evolving and we hope for new methods that can make peer review more efficient and sustainable.
References:
1. Seeber, M. and Bacchelli, A., 2017. Does single blind peer review hinder newcomers?. Scientometrics, 113(1), pp.567-585. https://doi.org/10.1007/s11192-017-2264-7
2. Van Rooyen, S., Delamothe, T. and Evans, S.J., 2010. Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ, 341, p.c5729. https://doi.org/10.1136/bmj.c5729
3. Polka, J.K., Kiley, R., Konforti, B., Stern, B. and Vale, R.D., 2018. Publish peer reviews. Nature 560, 545–547. https://www.nature.com/articles/d41586-018-06032-w
4. ASAPbio. Open letter on the publication of peer review reports. http://asapbio.org/letter. Accessed Feb 15, 2019.
5. Kovanis, M., Porcher, R., Ravaud, P. and Trinquart, L., 2016. The global burden of journal peer review in the biomedical literature: Strong imbalance in the collective enterprise. PLoS One, 11(11), p.e0166387. https://doi.org/10.1371/journal.pone.0166387