Citation beyond the numbers
Bob Schijvenaars explains why understanding context is the next frontier in research integrity
Citation has always been one of the foundations of scholarly communication.
It acknowledges previous work, provides evidence for new claims, and allows research to build progressively on what has come before. But as the volume of research continues to grow and concerns around research integrity intensify, it has become increasingly clear that simply counting citations is no longer enough.
One area that has received growing attention is self-citation. At first glance, it might appear to be a straightforward issue: simply identify how often authors cite themselves and flag anything above a certain threshold.
Researchers naturally build on their previous work. In many disciplines, citing earlier publications is entirely appropriate and often essential for establishing continuity within a research programme.
The challenge arises when self-citations become excessive or are used without genuine scholarly justification. Distinguishing between legitimate and questionable self-citation requires understanding not simply who is being cited, but why they are being cited.
That distinction is what makes this such a fascinating technical challenge.
Throughout my career, I have been interested in improving the quality and accessibility of scientific research. My background is in Medical Informatics, and I have spent much of my career working at the intersection of technology and scholarly communication, from journal recommendation systems to search technologies and machine learning applications that help researchers navigate an ever-expanding body of literature.
Today, as VP of Data Science and Infrastructure at Digital Science, I work closely with both customers and engineering teams. It is a position I particularly enjoy because it allows me to validate ideas directly with publishers while remaining grounded in what current technology can realistically deliver.
From concept to practical tool
For many years, research integrity has been moving steadily up the agenda for publishers. As publishers strengthen editorial screening and integrity processes, it became increasingly apparent that citation behaviour deserved closer attention. The question was no longer whether problematic citation practices existed, but whether they could be identified accurately and efficiently before publication.
The breakthrough came from combining advances in natural language processing with robust publication metadata.
Our colleagues at Writefull developed an early proof of concept demonstrating that machine learning models could identify not only self-citations but also understand the role each citation played within the surrounding text. That early work showed genuine promise. My team then focused on transforming the concept into a production-ready system capable of operating reliably at publisher scale while continuing to improve its accuracy.
The result is Dimensions Citation Check.
Looking beyond citation counts
From an editor’s perspective, the process is deliberately straightforward. A manuscript PDF can be uploaded exactly as received, without any reformatting or additional preparation. The system extracts the references, resolves author identities using the Dimensions database, and compares the authors of the submitted manuscript with those of every cited publication.
Author identity resolution is far more important than many people realise. Simple name-matching is unreliable, particularly when common names are involved. Without robust author disambiguation, false positives quickly become a significant problem. By leveraging the curated author identities within Dimensions, Citation Check can determine accurately whether a cited paper genuinely represents a self-citation. Equally important is reliable reference resolution, which presents its own technical challenges and benefits from the same underlying infrastructure.
Once self-citations have been identified, the analysis moves beyond counting. Each citation is located within its precise sentence, allowing the surrounding context to be examined. The system assesses whether the citation appears substantively justified or whether it displays potentially problematic characteristics. These may include citation stacking, where numerous self-citations support a single statement, or instances where authors appear to use their own previous work as external validation.
Rather than replacing editorial judgement, the system provides evidence. Editors receive an overall risk classification, alongside supporting information that includes self-citation rates, author-level breakdowns, sentence-by-sentence analysis and an assessment of whether individual citations appear appropriate. The analysis is completed within minutes and can be run across multiple manuscripts simultaneously, enabling editors to investigate concerns early in the submission process without introducing unnecessary delays.
This early screening capability is perhaps one of the greatest practical benefits for publishers. Editorial teams are under increasing pressure to process growing submission volumes while maintaining rigorous quality standards. When Citation Check identifies potentially problematic citation behaviour, editors have the opportunity to investigate before committing substantial time and resources to peer review or other costly stages of the publication process.
The future of citation integrity
Although manuscript screening is the immediate application, I believe the longer-term opportunities are even more exciting.
Large-scale analysis of published literature could help identify emerging patterns in citation behaviour across disciplines, institutions or collaborative networks. It may become possible to detect citation rings, identify unusual organisational self-citation patterns, or uncover broader trends that would be invisible when examining individual manuscripts in isolation.
As the scholarly publishing community continues to invest in research integrity, I believe greater scrutiny of citation quality will become an increasingly important part of editorial workflows. Citations should strengthen scientific arguments, not artificially inflate metrics or create misleading impressions of authority.
Ultimately, technology should not replace human editorial judgement. Instead, it should provide better evidence, surface potential concerns more efficiently, and allow editors to focus their expertise where it matters most.
If we can help publishers make more informed decisions while improving confidence in the published literature, then we will have made a meaningful contribution to research integrity.
Bob Schijvenaars will be presenting Dimensions Citation Check at a live webinar on Monday 27th July. Register here.
Bob Schijvenaars is Senior Vice President Technology at Dimensions