Behind the headlines are exciting initiatives that have the potential to, not just improve peer review, but optimise it for 21st century scholarship, says Eric Hall
If you have been following the recent scandals involving peer review rings, falsified research data, sloppy editorial practices, reviewing 'services', and paper mills, you can be forgiven for thinking that peer review is broken.
However, behind the headlines are exciting new initiatives that have the potential to not just improve peer review but optimise it for 21st century scholarship.
Here, then are four reasons to feel good about the future of peer review.
1. Research indicates that the vast majority of researchers still value and trust peer review. A 2013 study conducted by the University of Tennessee and CIBER Research compiled data that 'points to the fact that peer review is in fact the central pillar of trust(1)'. The 2014 Author Insights Survey sponsored by the Nature Publishing Group and Palgrave Macmillan found that the quality of peer review was ranked third – right behind journal relevance and reputation - in the list of factors that authors consider when submitting their research to a scholarly journal(2).
Sense about Science, a strong advocate for improving peer review, surveyed more than 4,000 researchers and found that 'almost all researchers (91 per cent) believe that their last paper was improved as a result of peer review,' and 'most (84 per cent) believe that without peer review there would be no control in scientific communication.(3)' Strong endorsements indeed. However, despite the support for peer review, many respondents felt there is room for improvement. As one study put it, 'While there was a strong attachment to peer review, most researchers in the focus groups prefaced their expression of trust with recognition that there were problems with the way it was undertaken. They were not blindly trusting of peer review.(4)' So, although peer review remains the preferred method for evaluating scientific research, changes must be made to improve and increase trust in the process.
2. More scholars are looking at peer review as a scientific object of study. It is ironic that the one process we use to validate scientific information is not very scientific, nor is it very well understood. Fortunately, there are signs that this is changing. The role of technology in the peer review process has enabled better collection and analysis of quantitative data while giving researchers insight into the qualitative aspects of peer review. These include the role of anonymity in peer review; peer review methodologies and measurement; and gender & judgment bias. Below are just a few examples of some of the research published in the last couple of years that focused exclusively on peer review:
Nicholas, D., Watkinson, A., Jamali, H.R., Herman, E., Tenopir, C., Volentine, R., Allard, S., Levine, K. (2015) Peer review: still king in the digital age. Learned Publishing, 28, 15-21. doi: 10.1087/20150104.
Etkin, A. (2014). A new method and metric to evaluate the peer review process of scholarly journals. Publishing Research Quarterly. doi:10.1007/s12109-013-9339-y
Faulkes, Z. (2014). The vacuum shouts back: Postpublication peer review on social media. Neuron. doi:10.1016/j.neuron.2014.03.032
Nicholas, D., Watkinson, A., Volentine, R., Allard, S., Levine, K., Tenopir, C., & Herman, E. (2014). Trust and Authority in Scholarly Communications in the Light of the Digital Transition: setting the scene for a major study. Learned Publishing, 27, 121–134. doi:10.1087/20140206
Onitilo, A. A., Engel, J. M., Salzman-Scott, S. A., Stankowski, R. V, & Doi, S. A. R. (2014). A core-item reviewer evaluation (CoRE) system for manuscript peer review. Accountability in Research, 21, 109–21. doi:10.1080/08989621.2014.847664
Paolucci, M., & Grimaldo, F. (2014). Mechanism change in a simulation of peer review: from junk support to elitism. Scientometrics, pp. 1–26. doi:10.1007/s11192-014-1239-1
Park, I.-U., Peacey, M. W., & Munafò, M. R. (2014). Modelling the effects of subjective and objective decision making in scientific peer review. Nature, 506, 93–6. doi:10.1038/nature12786
Vesnic-Alujevic, L. (2014). Peer review and scientific publishing in times of web 2.0. Publishing Research Quarterly. doi:10.1007/s12109-014-9345-8
3. There are now more organisations devoted to understanding and improving peer review than ever before. Peer review training and best practice has typically been the domain of publishers. But now there are several independent organisations working hard to increase understanding of peer review and optimise the process for 21st century scholarship.
Sense about Science (www.senseaboutscience.org) has become a rich resource for anyone interested in peer review research, training, and policy. In 2004, they formed a Working Party that published a comprehensive study entitled Peer Review and the Acceptance of New Scientific Ideas(5). Their Voice of Young Science program holds regular peer review workshops and their ‘Nuts and Bolts’ guide for early career researchers is a great summary of what is currently known about peer review(6).
Sense about Science has become a leader in peer review and is working hard to fulfil the demand for more information and training. In 2014, COST (the intergovernmental framework for cooperation in science and technology in Europe) formed a group representing the first government-funded, multi-national effort to 'improve the efficiency, transparency, and accountability of peer review.' This effort, known as PEERE (www.peere.org), reflects the importance of peer review to researchers as well as to funding and governmental agencies. One of their key deliverables will be a peer review framework including principles, guidelines, indicators, and monitoring activities designed to enhance our understanding of what works and what doesn’t in scholarly communication. Sense about Science and PEERE are only two examples of organisations that are answering the call to improve the peer review process for the benefit of academia and society.
4. There are several new products and services designed to preserve and enhance peer review. To meet the growing demand for solutions to some of the problems facing peer review, several companies have devised new products and services. PRE-val is one such product. PRE-val is the only service to provide independent, third-party verification of the peer review process at the journal and article level. PRE-val leverages metadata provided directly from the manuscript tracking system (Ed. Manager, ScholarOne, etc.) to confirm that a paper has undergone peer review in the manner advertised, and parse and display information related to the peer review process. Confirmation is provided and information is displayed via the PRE-val badge (Please visit www.pre-val.org for a detailed demo).
At its essence, PRE-val seeks to unlock the ‘black box’ of peer review and provide more information about the journey each paper takes prior to publication. PRE-val benefits each of the key stakeholders in the scholarly communication chain. It provides researchers with a visual indicator of quality and empowers them to make their own decisions about what level of peer review is adequate for their purposes. PRE-val gives Editors the opportunity to promote their peer review process, acknowledge the hard work of reviewers, and encourage submissions from authors looking for rigorous peer review.
Publishers also benefit from the PRE-val service by obtaining third-party, independent validation of their peer review process, reinforcing trust in their content and brand, and distinguishing legitimate OA journals from predatory titles. PRE-val is an idea whose time has come.
The recent scrutiny of peer review has uncovered some worrying trends. Fortunately, there is a small army of individuals and organisations working hard to better understand peer review and how it can be improved to fit the demands of today’s scholars. By supporting their efforts and promoting their work, we can ensure peer review remains the cornerstone of scholarly communication for many years to come.
Eric Hall is director of business development for PRE (peer review evaluation) at Striatus.