Tracking and validating reviews

Share this on social media:

Topic tags: 

Childhood friends formed New Zealand-based Publons to help academics build a reputation for their reviews, writes Sian Harris

When Andrew Preston and Daniel Johnson were boys riding their bikes and playing football in their home town of Whakatane, New Zealand, they had probably never heard of peer review. They certainly didn’t imagine that one day they would form a company together that aims to improve the process.

Nonetheless, that is exactly what happened. The result is Publons, an open-access peer-review platform where academics can build reputation for reviewing, rating, and discussing academic research.

The idea for the start-up came about when Preston was doing post-doctoral research in physics at Boston University, USA. He and Johnson, a self-taught software developer, were discussing the challenges of research communication. They talked about Stack Overflow, a website that provides free questions and answers for programmers, and wondered why nothing similar existed in scholarly publishing.

Out of this came the idea of doing something themselves. As Johnson explained: ‘Our ultimate aim is to speed up science and improve the slow and inefficient peer-review process. We see that reviewers don’t have any incentive to do reviews.’

There are two sides to Publons’ approach. The first is to improve post-publication peer review. According to the company, ‘post-publication reviews will no longer need to spend months or years working their way through the literature, but will instead become part of an ongoing conversation, immediately available and accessible to anyone, anywhere.’

The second aspect of Publons’ work is to give credit for pre-publication peer review, to get information from journals about who does reviews and to allow reviewers to share this information on their websites. The aim is to act as a central hub for journals and authors, recording all peer review whether it is done through Publons or elsewhere and building up author profiles.

This approach, says the company, will ‘give pre-publication reviewers credit for their work, providing positive incentives to produce high-quality reviews, and giving journal editors a large pool of excellent reviewers.’

Johnson said: ‘A lot of people spend a lot of time evaluating research but these contributions are rarely captured. Publons is not about trying to get people to do more work but about allowing them to get more recognition.’

He added that the company is considering an overall score for reviewers that takes into account public review, blind review, and feedback on reviews. ‘The opportunities are enormous,’ he observed.

The company was formed in February 2013 in Wellington, New Zealand. It was accepted into a local business acceleration programme, which provided some seed funding and the opportunity to pitch to investors. From this, Publons raised a round of seed funding that enabled it to employ some more people. Now the business has seven employees.

‘We are still at the early stages but are in talks with a couple of journals. It is a rather large challenge for journals to find good reviewers and we want to help them perform their services more efficiently,’ said Johnson. ‘The revenue model is not to charge researchers anything but to provide services to journals and publishers.’ He added that the exact charging model is still to be determined but will possibly be on a per-search or per-month basis.

Of course such things are never as straightforward as they might sound. As Johnson observed, ‘there are 101 challenges with building a platform from scratch; number one is probably building the community. We are not fooling ourselves it will be easy. It is a change in culture but that’s the way the world is going.’

Steps to success

Nonetheless, towards the end of last year Publons made some significant announcements towards its goals. First came the news that it would begin allocating digital object identifiers (DOIs) to post-publication peer reviews.

According to the company, although DOIs are standard for published articles and datasets, the application of DOIs to peer review is a first. Preston, Publons’ CEO, explained the reasons behind the move: ‘Reviewers’ valuable comments and observations are now given the credit they deserve, and this provides an incentive for reviewers to provide high-quality reviews, improving both the underlying science and commentary.’

‘Once you have a DOI it is citable and acts as a badge. One of our users has already started citing his post-publication review discussions in other papers,’ said Johnson. The assignment of DOIs means that academics can adds reviews they have done to publication records and therefore applications for funding and jobs.

This development was followed by an initiative to also give researchers the ability to build a portfolio of their pre-publication peer review work, along with the opportunity for journals to validate the quality of these peer reviews.

According to the company, pre-publication peer reviews will not, for the most part, be published; the existing types of pre-publication peer review – including closed, blind and double-blind – will be maintained. However, the initiative will mean that reviewers can show their activity in the academic community, and be given credit for it.

Publons is also providing academic journals with the ability to ensure and show that all pre-publication peer reviews are validated. This should enable journals (both open access and paywalled) to prove the quality of the peer reviews behind their published papers.  It will also enable pre-publication reviews to become an official part of researchers’ resumes.

The MacDiarmid Institute for Advanced Materials and Nanotechnology, a national network of New Zealand’s scientists in these fields, is already considering the use of the Publons platform as the source of record for academic peer-review reporting when used in hiring and promotion decisions. Other organisations have also expressed interest, according to the company.

So, what is it like starting a company with a childhood best friend? ‘I guess it’s not for everybody,’ said Johnson, ‘but over the years we have got very good at having disagreements that lead to good results.’