Moving to the Mainstream
Rebecca Pool asks: was 2017 the year that scholarly publishing let in open peer review?
Peer review forms the crux of good science, yet few would argue it works well.
Ensuring research quality, tracking down competent reviewers and avoiding peer review manipulation are just a few of the problems that have left many in scholarly publishing searching for a new way.
Enter open peer review; a transparent alternative that includes peer review reports being published alongside the research article. And from BioMed Central, PeerJ and F1000 Research to Springer Nature, Elsevier and The Royal Society, more and more publishers are taking part.
Andrew Preston, co-founder and chief executive of peer review service provider, Publons, recently acquired by Clarivate, reckons 2017 was the first year that he actually saw a marked shift towards transparent peer review. He points out how the 2017 Peer Review Week – focusing on transparency – was ‘remarkably bigger’ than past years, with the US-based Peer Review Congress also reflecting the enormous interest in peer review right now.
But, importantly, Preston also believes interest from industry’s traditional publishing heavyweights is on the rise. As he highlights: ‘We’re seeing a number of publishers talking publicly about the possibility of publishing reviews or allowing reviewers to sign their name to reviews now.
‘Elsevier, the largest publisher in the world, has been discussing a move towards transparent publishing for several years, and I think in the near future we will see more major publishers exploring similar [ventures],’ he adds. ‘Behind the scenes, a number of major publishers are now engaging with us to expand the scope of their transparent programmes... and these companies have shown particular interest in the last 12 months.’
Indeed, a little more than a year ago, Elsevier unveiled plans to add optional open peer review to its fleet of 2,500 journals by 2020. The company had initially published editor-selected peer review reports alongside articles in Agricultural and Forest Meteorology as early as 2012.
Positive reviewer feedback led to the ‘Open Peer Review pilot’ being rolled out across several additional titles in 2014, with reviewers having the choice to opt in. Resounding acceptance prompted the decision to scale up the scheme.
‘We surveyed 20 reviewers and 95 per cent said the availability of open peer review would not influence their decision to recommend rejecting or accepting an article,’ highlights Philippe Terheggen, managing director, journals, at Elsevier. ‘This is a very convincing, high number.’
Clearly the infrastructure challenges relating to large-scale open peer review roll-out are significant, but the company has already adapted Science Direct and its underlying production systems to host peer review reports. What’s more, it is currently updating its editorial and general peer review systems to process the reports, and expects to complete all required infrastructure changes by the end of 2019.
‘We will use our positive survey results to show our journal editors how beneficial open peer review can be, but we will not mandate it,’ emphasises Terheggen. ‘In some of our higher segment journals, such as The Lancet and Cell, we will be more cautious as [editors] have raised questions over its impact, but we are very excited.’
Indeed, across the board Terheggen is seeing interest in transparent peer review. Elsevier’s Open Peer Review pilot was trialled across an eclectic mix of journals, covering subjects from fracture mechanics and hydrology, to medicine and agriculture, with success in each discipline.
‘We see acceptance across all disciplines and other publishers are seeing the same picture,’ says Terheggen. ‘I really don’t see a lot of differentiation now and I also don’t see many risks. Open peer review provides an extra layer of information and context to research articles and this will help research.’
Elsevier’s Philippe Terheggen feels open peer review is gaining more support
Clearly for Publons’ Preston, information is also very important to the peer review processes. As he highlights: ‘More transparency in the peer review process means that authors, reviewers, readers and funders can all make more informed decisions about a particular journal and its research,’ he says. ‘Publons will provide as much information as a journal is happy for us to provide, whether it’s an entirely open review or you can just see which of your colleagues are reviewing for that journal.’
‘We also make it possible for people to see if a journal has been endorsed for having a good peer review process,’ he adds. ‘We really want users to browse all of this information on Publons to get an idea of who in the community is engaging with a journal.’
A different tack
In September 2016, Springer Nature’s open access journal, BMC Psychology, decided to test the thorny issue of publication bias by withholding research results from peer reviewers in a randomised controlled results-free peer review trial. As part of this, selected reviewers did not to see the results, discussion and conclusion sections of manuscripts before deciding whether or not to accept these submitted articles.
As Katherine Button, from the University of Bath, a transparent research advocate told Research Information at the time: ‘The current system favours publication bias, as significant results are seen as more important by publishers, academics and the system in place that measures performance. Results-free peer review really isn’t a new idea but it hasn’t ever gained traction, which is odd, as it is such an easy win if it does serve to reduce this bias.’
Fast-forward to June 2017 and the first BMC Psychology article to undergo results-free peer review was published with positive feedback from the authors. Today, any author submitting an article to the journal can opt into results-free peer review. And, what’s more, industry developments suggest this flavour of peer review is gathering momentum.
Late last year Elsevier also started trialling ‘results-masked review’ in its Journal of Vocational Behaviour. In a similar vein to Springer Nature, the publisher hoped to level the playing field for research data and allow an article to be judged on the merits of its research questions and methodology, not its findings.
‘In the field of psychology, there was a call to increase the rigour of science; the intention was to improve the research, rather than the article,’ highlights Terheggen. ‘It was recognised that increasing the rigour of peer review would help, and results-masked review followed.’
As with Springer Nature, success ensued, and as Terheggen said: ‘[This type of peer review] was received very positively and we are now in the process of rolling it out across a handful of journals. As with open peer review, results-masked review is not something that we want to mandate but we have seen that it does increase the quality of the research and the article, which is beneficial to this community. Whatever method of peer review one takes, we strongly believe that high-quality peer review is fundamental to further increasing the relevancy and impact of the research we publish.’
For Terheggen, a key benefit of Elsevier’s open peer review, is that it will support its review recognition platform, set up to help editors find reviewers and also to provide reviewers with recognition. Like Publons, and many publishers, Elsevier has long-realised the importance of providing reviewers with recognition for their efforts.
In 2015, the publisher established a platform through which reviewers can register and volunteer their services to particular journals. This enables less well-known researchers to deploy valuable knowledge on that publication. And crucially, the platform also provides ‘perks’ such as peer review certificates, and a profile reflecting the importance of a reviewer’s contributions.
Elsevier’s Journal of Molecular Biology, for example, has seen astounding take-up of the platform, with the number of registered reviewers increasing by 700 per cent in just one year. ‘We now have 500,000 reviewer profiles across the entire platform, and all of our figures indicate that it is very relevant to provide reviewers with recognition,’ highlights Terheggen.
‘The reviewer recognition platform applies to most of our journals but a key goal of the programme is to give reviewers recognition and we think that rolling out open peer review on a larger scale can only support this.’
Indeed, Preston is excited about the rapid growth of Publons in the past year, outlining how the platform is now home to a quarter-of-a-million reviewers and some 1.5 million reviews. ‘The fact that this has grown so quickly over the last year signals that the market, as a whole, is interested in transparency,’ he says.
Looking to the future, Preston is also excited about how the Clarivate acquisition has provided his company with a clear opportunity to scale its activities and bring change more quickly to peer review.
‘By combining Web of Science data with Publons reviewer data we can provide a more comprehensive view to editors when making a decision about reviewers,’ he adds. ‘And we think we can also work with more institutions and funders around the world to provide more information on who is reviewing, so they can also make more informed decisions.’
And for Elsevier’s Terheggen, change could come sooner rather than later. ‘There is a cautious move towards acceptance of transparent peer review and open peer review is a part of this,’ he says. ‘The world is not entirely ready for open peer review on all journals, but if that moment came in the next five years, I wouldn’t be surprised.'
Tackling fraud
As the move from traditional to transparent peer review gathers momentum, many open peer review advocates believe transparency can go some way to stave off industry issues such as predatory publishing and fake peer reviews.
Having peer review reports ready to read next to a published paper can help researchers distinguish between reputable and not-so-reputable journals. And removing the secrecy that shrouds traditional peer review could stem peer review manipulation and bogus reviews so widely publicised of late.
Elsevier’s Philippe Terheggen believes transparency can only help, saying: ‘Open peer review is certainly an instrument to increase trust in an article. For instance, you can see this in articles that are publishing original research data, as these get cited more than the average article. Any relevant information added to a research article increases trust in that article.’
Publons’ Andrew Preston also reckons transparency can help to alleviate predatory publishing. But as he emphasises: ‘This really isn’t a black and white issue, there are all sorts of interesting areas of grey, making it very difficult for a researcher or funder to make a judgement on a particular publication.’
Evidence for change
A recent study led by Dr Tony Ross-Hellauer of the Know-Center, Austria, revealed that 60.3 per cent of researchers believe open peer review should be mainstream practice.
Survey on Open Peer Review: attitudes and experience amongst editors, authors and reviewers, surveyed 3,062 researchers as Ross-Hellauer and colleagues tried, in part, to better understand what researchers understood open peer review to be, and also to glean information on attitudes and experiences.
‘With peer review becoming quite a major pillar of the Open Science agenda, we started looking into it and realised that there was quite a lot of disagreement on exactly what open peer review is,’ explains Ross-Hellauer. ‘We found that usually when people are talking about open peer review, they mean open reports, or open identities, or both.’
As well as revealing that a little more than 60 per cent of those surveyed favoured open peer review, results indicated that respondents were also in favour of other areas of open science, such as open access (88.2 per cent) and open data (80.3 per cent). And, interestingly, as Ross-Hellauer tells Research Information, respondents were heavily in favour of increased interaction between reviewers, as well as between reviewers and authors.
‘More than two thirds [of those surveyed] thought this kind of interaction and less mediation from the editor would improve peer review,’ he says. ‘From our comments, there was an underlying feeling that this would make peer review more constructive and less oppositional.’
And while the 60.3 per cent majority favoured open peer review, most were reluctant about opening reviewer identities to authors. ‘The only way I can see that opening up peer review reports would be negative, was if you made this dependent on having open identities, as this really does not seem to be favoured yet,’ highlights Ross-Hellauer.
‘There’s a feeling that negative consequences, such as problems with tenure, may follow if negative comments were made to, say, somebody of a higher status.’
