Preprints and open peer review come of age

Share this on social media:


Scott Edmunds

Scott Edmunds highlights opportunities to improve the publishing process by improving transparency, accountability and speed

As editor in chief of GigaScience, for a decade we have increased transparency and trust in our published work by throwing open the entire peer review process, and letting people really 'look under the hood'. While this isn’t a new concept, the first experiments in open peer review go back at least 40 years and the medical community started embracing it in the late 90s, in the last few years the practice has become increasingly mainstream. Many Nature Journals and PLOS have taken the leap in opening their review process, and the UNESCO Recommendation on Open Science specifically highlighted open peer review as one of the approaches its member states should promote for open science. The challenges from the Covid-19 pandemic makes these moves even more timely, as increasing transparency in research is an extremely useful weapon in fighting scepticism in the scientific process during these turbulent times.

As peer review has opened, reviewers are now able to gain credit for their hard work, and technological approaches are now available to help them advertise and amplify their peer reviewing duties. On top of third-party platforms that capture peer review history, such as the pioneering Publons, ORCID has provided peer review functionality and a ‘Reviews Activity’ section on its profile pages. Leveraging this, journals such as GigaScience, F1000Research and PeerJ have been giving their peer reviews DOIs to make them independently citable and easily claimed in author ORCID profiles.

While there have been some false starts in attempts at creating a currency and ecosystem of 'portable peer reviews' by start-ups such as Rubriq and Axios, the rise of preprint peer reviews has made this a more community and researcher-driven effort. After early but unsuccessful efforts of third-party systems like Academic Karma (rebranded as before closing, and something we found way ahead of its time), the preprint platforms themselves have taken the logical step of moving beyond simple commenting systems like Disqus to more sophisticated and integrated linking of full peer reviews, and these have finally started to gain momentum. 

In 2019, Cold Spring Harbor Laboratory released their TRiP: Transparent Review in Preprints (TRiP) functionality, which enabled journals and peer review services to post related peer reviews on their bioRxiv preprint server. The first participating members were open-peer review publishers eLife and EMBO Press, alongside a survivor of the first wave of portable peer review services, Peerage of Science. GigaScience and our new journal GigaByte, which mandates posting preprints of manuscripts, have now joined this service, alongside a growing ecosystem of community peer reviewing sources. Also contributing to TRiP, Review Commons is a new platform for journal-independent peer-review in the life sciences, transferring on behalf of participating authors the refereed preprints to bioRxiv and onwards to the17 affiliate journals currently signed up to the scheme, including TRiP founding partners EMBO Press and eLife, as well as ASCB, The Company of Biologists, Rockefeller University Press and PLOS.

Powered by the online annotation tool, TRiP works by posting peer reviews in dedicated groups alongside relevant preprints on the bioRxiv and medRxiv websites. While an excellent addition to the publishing arena, this system is embedded in the two CSHL preprint servers and currently doesn’t work for the many other preprint platforms operating outside of biology and medicine. Additionally, the lack of an automated ingest system makes adding reviews to the preprints less efficient; however, eLife has been investing in a new and more automated platform called Sciety that aims to streamline, evaluate and curate preprints from a wide array of places in one place. The preLights preprint highlighting service and the new PeerRef preprint peer reviewing service are already included among a number of feeds and groups, including many of the previously mentioned schemes. 

Another community driven and not-for-profit effort feeding in here is the Peer Community In (PCI) initiative. This has created thematic communities of researchers that review and recommend articles posted on preprint servers and other open-access repositories. Alongside a network of 90 'PCI-friendly' journals that are willing to consider these reviews, they have their own community-based diamond open-access journal Peer Community Journal that publishes articles recommended by PCI.

The challenges of balancing speed versus the trustworthiness of COVID-19 preprints has been a major point of discussion over the last two years, and the ability to peer review them in this manner means these technologies have arrived at exactly the right time. Sciety is gathering third party evaluations of papers and preprints from the Novel Coronavirus Research Compendium from Johns Hopkins University, and Rapid Reviews COVID-19 overlay journal from MIT press.

An added benefit of the rapid growth of preprint peer reviews is not only that it helps authors and readers, but also, with the identity of the reviewers included, it provides a wider platform for journals to find new reviewers and editorial board members. ASAPbio has taken a proactive step in making this happen by launching the Preprint Reviewer Recruitment Network (PPRN) to enable willing researchers to share preprint feedback as work samples for review by participating journals. The rationale behind the scheme is the desire from early career researchers to participate in peer review, and this is an excellent way to help them break into reviewing or editing roles. While still a pilot, this is in the process of being set up as an ongoing project. As a participating journal, we’d encourage more journals and potential reviewers to sign up to this important effort.

While preprints, open peer review, and portable peer review have had many fitful starts and stops over the last few decades, their time seems to have finally arrived. As standalone movements they were a harder sell, but now, working together, they interact and support each other synergistically, and the UNESCO endorsement of Open Science has helped make these concepts mainstream. New technological solutions have made it much easier to set up and run these workflows; through their integration into fundamental publishing infrastructure and standards via PMC, ORCID, the STM Association and NISO; as well as new open platforms from publishers (via funders moving into this space such as Chan Zuckerberg Initiative). Research culture has also been changing, with younger researchers seeing transparency and openness as a norm, and funders and publishers endorsing and promoting such efforts. With the COVID-19 pandemic being the final driver for people to look to new mechanisms to disseminate research more quickly and in a more transparent and trustworthy manner. 

Our own personal experience of all of these many schemes has been very positive, with more scrutiny, faster communication and fairer and more constructive discourse coming as a result. We would recommend authors, reviewers and journals test out these many pilots and platforms, and look forward to further developments in this now flourishing ecosystem.

Scott Edmunds is editor in chief at GigaScience Press