The art of judging things for what they are

Share this on social media:

Image: imtmphoto/shutterstock.com

Kim Eggleton on how using anonymity can create more diverse, equitable, and inclusive peer review

Every minute of every day we make judgements. Shall I answer this call? Can I trust a stranger to hold the lead of my dog whilst I do up my laces?  Judging our environment often based on what we already know or on previous experiences. But what if we were to strip out identity, would that improve our judgement?  

As part of the peer review process, many academics are asked to judge the quality of research submitted for publication. Together with publishers, they act as the gatekeepers for research, forming the backbone of scientific publishing since science records began in the seventeenth century.  

Academics are humans and they have opinions about not only the work they need to assess but also about the people who wrote the work. Studies show that academic reviewers are unconsciously more likely to accept work if it is written by people they consider to be like them – this seems to apply across gender, ethnicity and geography.  

We know that if our editorial boards and our reviewers are predominantly made up of white men from the US and Europe, that this could be influencing our publishing decisions. Perhaps people that don’t reflect a certain demographic are being disadvantaged, because of our unconscious bias to favour people like ourselves? To diversify scientific publishing, we not only need to give attention to which voices we are hearing, but also look at who is making judgments about those voices. As publishers, we have a role to play to ensure our editorial boards and reviewers represent the diverse community they serve, and that our staff are aware of this bias too and get the training they need.  

Conscious effort can really help mitigate against bias. We’ve been working hard this year to diversify our boards and invite a more diverse range of reviewers, partly for the reasons above, but also because we know reviewers are overburdened and declining more invitations. According to our 2020 Peer Review report, 40 per cent of German, US and UK reviewers say they receive too many review requests. As a result of our efforts, we are seeing the average number of reviewers invited per paper reduce for the first time in years. We are ahead of the global trend for female authorship in physics, with 22 per cent of our papers accepted for publication being from women, compared to a global average of just 17 per cnt in 2016.  

Another step we’ve taken is the introduction of double anonymous peer review for all our journals. In the last year we’ve been moving all our journals over to double-anonymous peer review – where the reviewer and author identities are concealed – and the feedback from authors and reviewers is overwhelmingly positive.  

It’s encouraging to see that double anonymous is not only preferred by early career researchers or people who might feel that their name or geographical area of residence would reflect negatively on the assessment of their work. Recently a Nobel laureate anonymised their manuscript on submission to our journal for double anonymous peer review, demonstrating a belief in the publishing system and a trust in the quality of the research rather than solely on reputation recognition.  

However, double anonymous is not a panacea. To tackle inequality and lack of diversity in scholarly publishing we need a holistic approach. Rather than creating echo chambers, we need people from different disciplines and across industry to come together to discuss best practice.  

As part of peer review week, we’re organising a webinar which will be held this Wednesday the 22 September at 3pm BST.  Anyone can come along to listen and participate in the discussion about the barriers researchers with different identities can face when trying to get involved in peer review and how identity in peer review can affect the quality of research and ways to improve equality and diversity in peer review. 

We believe that our contributors should reflect the diversity of the physical-sciences community, and we recognise that there are inequalities within peer review across the science, technology, engineering and mathematics (STEM) subjects. We’re all human and there’s still a lot of work to be done, but we know from research that diversity leads to better science. As a society publisher we will continue to advance physics for the benefit of all.