Experts give new slant on peer review

Share this on social media:

Sarah Greene has just been appointed editor-in-chief of Faculty of 1000 (F1000) as well as editor of The Scientist magazine. We ask her how post-publication peer review helps researchers guide scientists to the most important work

What are the limitations of the peer-review process?

Pre-publication peer review is still important but there is a lot of pressure now on the review process, given the enormous amount of research being submitted for publication today. Not only is the research more specialised, requiring enormous expertise from the reviewers, but it is impractical for reviewers to assess the often massive datasets that underpin the research. In addition, there have been studies indicating that traditional peer review has inherent biases that lead to rejection of truly valuable studies. And it’s a tremendous weight on the scientific enterprise in terms of time and expense.

How can this be addressed?

There is a movement afoot to make more information available, without the extensive formal review prior to publication and in a more timely fashion. Then the detailed examination would come after publication. In the case of F1000, there are 10,000 experts on board representing over 400 specialties, selecting the best of the research and providing evaluations according to how the papers impact their own fields. And you might get evaluations from several different specialties, alerting scientists to important information from many perspectives.

Post-publication peer review (PPPR) builds on open access, because it depends on research being widely available to everybody who might eventually have the impetus to test the data or repeat the methodology. PPPR serves to filter the massive amount of information based on expertise and knowledge at a given point in time. There are many different ways that research will be reviewed after publication – and be confirmed, found faulty with or had its conclusions modified or extended. The point is to get the content out in the open, as fast as possible, accessible to as many as possible, subject to the basic tenets of accepted scientific practice.

Aren’t your experts also biased?

We strive to keep bias out of the equation in a variety of ways. We require conflict-of-interest statements and our definition of conflict goes way beyond financial interests. We do not influence the reviewers with suggestions or recommendations of papers to evaluate. There is a continual influx of new faculty members to keep the selections and insights fresh. Most importantly, evaluations and comments are openly attributed to the faculty, and linked to their profiles, providing full transparency of their work to subscribers. This differs from the current peer-review system, which is generally a closed system with anonymous reviews.

With your numerical rating of papers, does F1000 compete with ISI’s Citation Index?

Citations can be a useful measure of a paper’s import, but a recommendation and rating from an expert on the faculty, or from a number of experts in different disciplines, provides an entirely different kind of measure.

There are many types of metrics that act as filters and measures of import, including pageviews and bookmarking. But F1000 has a different type of prestige because the faculty members are at the top of their disciplines, and the system cannot be gamed in the same way as anonymous quantification. So now, just being chosen for inclusion in F1000 is starting to be regarded as an honour for the authors and worthy of listing on their CVs.

Where does your role on The Scientist fit in?

The Scientist was founded in 1986 by Eugene Garfield, who created the ISI and the citation rankings. The ISI hot papers used to inform the content of the magazine but we are moving over to featuring our faculty-recommended papers instead. By getting these scientists to work with our staff and freelance journalists, we produce articles and profiles that reflect emerging trends and big ideas of science, but with narrative. I like to think of the Faculty of 1000 as the world’s best advisory panel.

What are the business models?

There is a subscription model for Faculty of 1000 and advertising for The Scientist. By combining the products, there will be new digital advertising opportunities. We have to compete for institutional budgets, but while traditional markets may be depressed at the moment, given the economic climate, we’re pleased that subscriptions overall are growing, as is advertising revenue.

Interview by John Murphy