Citations are more than just a number

Citations are much more than just a number or a metric used to price journal subscriptions or APCs, writes Josh Nicholson

Citations are primarily used today as a number. A high number is good, and a low number is not. This view might be an oversimplification, but we generally treat citations superficially, which is perhaps why many researchers find their use in research evaluation problematic. 

Citations, though, are pretty amazing if you think about them. They connect ideas and findings across generations and different disciplines. They are permanent yet dynamic, with new citations being made to papers hundreds of years old in some cases. They are like the neurons of our global mind. Despite their prominence in scholarly publishing, they have changed very little over time. The citation indices, Web of Science and Scopus, look pretty similar to what they looked like five or ten years ago. The use of citation indices has also not changed much. Most people using these tools do so relatively infrequently, primarily for bibliometrics research or evaluation, not for information retrieval or discovery. Indeed, ask an undergraduate or graduate student how often they use a citation index, and they will likely reply never or very rarely. 

However, this stagnation in displaying and using citations has not been for lack of want or effort. Various proposals and discussions have occurred over the years discussing new uses for citations and potential improvements. Eugene Garfield, the creator of the first scientific citation index, suggested in the ’60s adding citation markers that described the citation type to citation indices. More recently, researchers have put together a citation ontology, CiTO, describing the many ways researchers use citations. Publishers, like PLOS, have also tried to introduce such different citation types or what they call “rich citations,” but such initiatives never made it past the proof of concept stage. 

However, citations and their display are now being dramatically improved with tools like scite and Semantic Scholar. These tools display the citation context and automatically classify citations into specific types, such as those that provide supporting evidence or those deemed 'highly influential'. These citations, which I have called 'Smart Citations', don’t just improve citation indices; they change who can use them and how they are being used. Specifically, citations become something that can be used daily as a new lens to better understand and interpret findings in the context of other research. Moreover, these citation contexts can also be searched, allowing one to reframe the idea of citations altogether from a mere link to an expert analysis or interpretation of the paper. Accordingly, research comparing the abstract to citation sentences from citing articles found that the citation statements (called citances) contained 20% more information. 

Going further, what if instead of looking at citations to a paper, you could look at citations to a topic: SciRide Finder, a tool introduced in 2018, allowed users to search citation statements themselves. Unfortunately, the tool has not advanced from a proof of concept stage and is limited to only a small subset of citation statements.

Recently, we launched Citation Statement Search at scite. This tool extends the work of SciRide Finder and allows users to search over 900M citation statements from 26M full-text articles. Citation Statement search turns citations into a source of information, not just a superficial metric used for evaluation. As one librarian described it: 'Because you are searching within citation statements, you can occasionally get the answer you want immediately answered in the citation statement or context.'

As we rethink citations, we can learn that they are much more than just a number or a metric used to price journal subscriptions or APCs. They are a resource that can help educate and contextualise research, something we can all probably agree the world is in desperate need of right now.

Josh Nicholson is CEO and co-founder of Scite