All clear?

Andrew Preston describes how to make transparent peer review work at scale

Transparent peer review shows the complete peer review process from initial review to final decision, and has gained popularity with authors, reviewers and editors alike in recent years.

But few publishers implement transparent peer review at scale, as it can be arduous to make even small changes to complicated, established workflows. Despite these challenges, it can be implemented smoothly, in a way which works for authors, reviewers, editors and publishers.

Transparent peer review allows readers to access a comprehensive peer review history, including reviewer reports, editor decision letters and authors’ responses, alongside the published article. Each of these elements is assigned its own digital object identifier (DOI), which helps readers easily reference and cite the peer review content. Transparency makes the process accountable and gives recognition for the work done. It may also aid the teaching of best practice in peer review, and deter manipulation.

But submission and peer review workflows are complex, often touching on every point of the publication cycle. Changing just one aspect can have knock-on effects for others, resulting in some reluctance from publishers in adopting more transparent peer review approaches.

In 2018 Publons and ScholarOne (both part of the Web of Science Group), worked with Wiley to create a new workflow. This resulted in Transparent Peer Review: the first cross-publisher transparent peer review workflow, which offers all these benefits, complies with best-practice data privacy regulation, and ensures the individual preferences of authors, peer reviewers and journals are met.

So how does it work? Transparent peer review is a Publons workflow which uses the ScholarOne API. The reviews and scholarly discussion can be read on Publons, which is accessible via a link from the published article. We will deploy a badge/plug-in – coming soon – that will display a summary of review data from the published article (for example, indicating the number of reviews and decision letters). If the reviewer opts-in to sign their name through reviewer recognition (in the spirit of fully open peer review) all of this connects with the reviewer profile on Publons, ensuring they get full recognition for their hard work.

Transparent peer review is now used by Wiley, IOP Publishing, Emerald Publishing Group and The Royal Society. The fact that it is being used across a diverse group of well-respected commercial, society, scientific and HSS publishers demonstrates the flexibility of the workflows. In most cases, the workflow replaces traditional peer review, but in others, they have replaced manual, existing transparent peer review workflows developed in-house which may not have been practical at scale.

The inaugural partner on this project, Wiley, was early to innovate on transparent and open peer review. Their pilot started in September 2018 with just one journal, and grew by a further 10 journals in January 2019. Since then, the pilot has proven popular. A further 50 journals are set to join in September 2019.

Wiley has conducted some initial analysis over a three-month time frame. They found transparency to be a popular choice – of the 2107 authors who submitted to the journals during that period, a staggering 87 per cent chose transparent peer review. To counter common fears or misconceptions around implementing transparent peer review they also focused on seven journals which did not previously operate transparent peer review. They did not find any difference in the number of reviewers invited, nor did they find a meaningful difference in time taken during the peer review process, an important point given the challenge of review fatigue and apathy. (Although two journals did show a significant improvement: a quicker turnaround, by 20 days!)

Chris Graf, director for research integrity and publishing ethics at Wiley, explained: 'These results are very encouraging for the entire research community. We take research integrity immensely seriously at Wiley and see this new review initiative as a step towards making the publishing process more transparent, encouraging better review quality and informed decision-making.

'It will also dissuade manipulation of the peer review process, and provide reviewers and authors with greater visibility and recognition of their efforts. It’s heartening to hear that other publishers, working across a range of disciplines are now joining this initiative.'

The results reflect our research on the Global State of Peer Review, which found that reviewers believe more training, greater recognition, and better incentives will have a positive impact on the efficacy of the peer review process. Indeed, younger reviewers are more likely to review for journals with open peer review models than older reviewers.

Some 40 per cent of respondents under the age of 26 – the next generation of peer reviewers – say they are likely or highly likely to review for journals that make author and reviewer identities, and review reports public, compared with only 22.3 per cent of respondents aged between 56 and 65.

Of course, the single and double-blind peer review processes still dominate today, and many research communities may be slow to adopt more transparency. But real change often starts in small, incremental ways. We couldn’t put it better than Paul Kirschner, editor-in-chief of the Wiley Journal of Computer Assisted Learning, who said: 'While this sounds like a radical idea, and in certain respects it is, it’s a first step in the direction of real open science.'

Andrew Preston is managing director and co-founder of Publons, part of the Web of Science Group