A new approach to peer review

What are your thoughts on the peer-review system?

There is a saying that conservative people are fond of: if it’s not broken, don’t fix it. We hear this said about peer review. Publishers often say this in public – but it’s not the view of all editors, reviewers, or authors. Many have told us that the process requires a lot of management, and they express frustration with sloppy reviews that might be done at deadline, or with little effort.

Traditional peer review is not broken, but it is making troublesome noises. If you owned an old car you wouldn’t wait for it to break down before you fixed it. We are looking at how to keep the parts of traditional peer review that work, and fix the ones that don’t.

Peer review’s strength is its voluntary nature – reviewers do it for the love of their field. An editor who weighs the reviews and ultimately decides whether work should be published is also a key part of peer review. Another important thing is that pre-publication peer review continues to be done by peers, and not in public. Twitter is not suitable for pre-publication peer review, although public commentary is an important part of post-publication peer review, the process of science that goes on over decades.

What are you doing?

We’re changing key parts of the process, enabling reviewers to choose to engage when something really interests them and when they have the time to do a review. We provide one place to submit. The central idea is to get rid of the cycle of submit, reject and submit again.

We hope authors will submit manuscripts via Peerage of Science. Upon submission, the author decides deadlines for each stage of the peer-review process. Scheduled software then enforces these deadlines.

Any validated researcher (unless affiliated with the author or explicitly excluded from a given manuscript) can submit reviews. The first deadline is usually within two weeks. Within this time, authors, on average, receive 2.8 reviews – comparing well with the months generally taken for traditional peer review.

Editors in participating journals can track any peer-review process, anonymously. If they lose interest in a paper they can simply abandon tracking the paper, which means that there are no rejection letters or hurt feelings. Editors can make publishing offers at any stage, and authors choose whether to accept an offer, or export completed peer reviews elsewhere.

During the process we’d hope it would be triple blind – no-one knows the reviewer, even the editor, although editors can contact them and ask their identity. After the review is done, we encourage reviewers to disclose their identity and comments publicly. But anonymity is a complicated issue, and always everyone’s free choice. As a new researcher I felt I should sign reviews to keep myself accountable, but signing was also a way to make a name for myself.

Peerage of Science is free to scientists. The idea is that publishers pay for the process – in a similar way to the traditional system. However, the pricing model is different. Publishers can send an offer to any manuscript but they only pay if the author accepts their offer, which means that they don’t have to pay for the peer review of rejected papers.

What’s in it for the reviewers?

In Peerage of Science we’ve introduced the concept of ‘peer-review-of-peer-review’ as an obligatory part of the process. Reviewers evaluate each other. For every article reviewers get a PEQ (peerage essay quality) score between 1 and 5. A reviewer’s personal PEQ is a sample-weighted average of their PEQs for different reviews and this goes into their personal profile. This means that reviewers have a personal incentive to do a good job and meet deadlines.

We hope this score is something people will feel is valuable – and that publishers will ask for it when hiring editors. Evaluating other research does not necessarily involve the same skills as being excellent at research.

We are planning to introduce integration with the ORCID system. Researchers will be able to link to their ORCID ID in Peerage of Science. We hope that eventually people will be able to display their PEQ in their ORCID profile, and make it available for the many other services participating in ORCID too.

We are also launching a new journal, Proceedings of Peerage of Science. This invites reviewers to write short commentaries based on the reviews they have done and these are published with the attached PEQ. Reviewers get a peer-reviewed publication out of it.

Over time, when people emerge as very capable reviewers I hope their essays will become something that people want to read. This online-only open-access journal will be free for authors and readers.

How has it gone so far?

Since we launched we’ve had 75 submissions and we have 22 journals participating in Peerage of Science. We know of 13 papers reviewed though Peerage of Science that have now been accepted for publication in peer-reviewed journals, four of which were the result of direct offers via the system. We [Seppänen and the other founders Mikko Mönkkönen and Janne Kotiaho] are all ecologists and biologists, so that was a natural field for us to start in but we are open to other opportunities. We have some reviewers registered in mathematics, physics and medicine. It takes time to build and we’re finding out the hard way that things like marketing, to both researchers and publishers, are important.

Interview by Siân Harris