Why research analytics built on AI needs to show its working
Merely appearing useful – versus being reliable, credible or provable – are different things, writes Kate Byrne
In the past two years, the research analytics market has – like most markets at the moment – flooded with AI.
Every significant platform now claims it. The language is familiar: AI-powered, AI-driven, AI-enhanced. Most of them return something that at least looks useful, fast. We are quickly learning, however, that merely appearing useful versus being reliable, credible or provable – when the stakes really matter – are different things.
The question worth asking now is not whether a tool uses AI. It is what the AI is built from, and whether that foundation is sufficient for decisions that carry real institutional consequence.
The reliability gap
When people compare AI research analytics tools, the question they tend to ask is how much the system can produce on its own – how automated, how fast, how comprehensive.
A system can score well on that measure while doing nothing more than confidently recombining what it already knows. But speed is not the same as reliability, and when a tool cannot distinguish a sound conclusion from a plausible-sounding one – wrong outputs are indistinguishable from right ones – they arrive with the same confidence, the same formatting, the same apparent authority.
Research has always been judged by its reproducibility. Research strategy that cannot meet that same standard – that cannot show its working, trace a finding to source, or be interrogated by a second pair of eyes – does not clear the bar that research itself demands.
For high-level decisions about where to invest, how to position, who to collaborate with – decisions that will face scrutiny in a board meeting or a funder review – plausible is not enough. Provenance is needed.
Connected data, not interchangeable data
A question about where to build strategic capability in a specific field requires not just publication data, but joined-up knowledge of what has been funded, what has reached clinical translation, what is being cited in government policy, and where real-world attention is concentrating. Those signals together tell a different story from any one of them in isolation.
The connections between data types are where insight becomes possible – assembling them for a single question, using tools that do not integrate those sources, can take weeks of specialist work. Research analysts have described the experience directly: the challenge is “taking disparate data systems and trying to combine them, then trying to figure out what we can actually answer – because we’re trying to pull data that was never meant to go together.”
Most research analytics teams simply do not have that enough capacity, meaning that the questions that require it must either wait or go forever unanswered.
Domain expertise is not a default
Over a decade of working with research institutions on strategic analytical challenges, one consistent observation stands out: the value in this kind of work is not in the data itself. It is in the judgment about what to do with it.
Digital Science has spent over a decade delivering strategic analytical work – benchmarking, landscape analysis, expertise identification – for governments, funders and research institutions around the world.
What that work produced, beyond the reports themselves, was accumulated judgment: the patterns and commonalities in methodologies that surfaced over time, and an understanding of the data that only comes from years of working with it closely enough to know not just how to query it, but its personality, its caveats, and its failure-points.
Most AI research analytics tools have data and a language model. What they lack is that layer of encoded practice – and with it, any mechanism for knowing when they are wrong.
Who has had access to this, and why that is changing
That analytical practice has, until now, been available mainly to institutions with the resource to commission it directly. Institutions that can afford dedicated research intelligence functions, extended project engagements and specialist expertise have consistently had better intelligence to work with than those that cannot.
Many institutions are not in that position – and the research community’s ability to make well-informed strategic decisions has been unevenly distributed as a result.
At Digital Science, we are working on changing that. Dimensions Research Strategy, which launched in July 2026, is built on Dimensions and Altmetric combined – publications, grants, patents, clinical trials, policy documents and real-world attention signals, linked across more than five billion research objects.
Those connections between data types are where the insight lives: the ability to see not just what researchers have published, but what they have been funded to do, what has reached clinical translation, what is being cited in policy, and what is generating public attention – the whole research landscape, navigated with context. That picture is encoded with the analytical methodology our research strategy team has developed over a decade of real client work.
The two use cases at launch, benchmarking and expertise identification, are the questions research leaders ask most often – but they are the foundation of something larger. Research integrity, funding intelligence, research security and impact evaluation are all areas where the same approach applies: connected data, encoded expertise, full provenance.
The ambition is a platform that gives every research institution the strategic intelligence it needs, across every dimension that matters – on demand, to the same standard of rigour that has until now required a dedicated engagement to access.
Kate Byrne is Senior Vice President of Product, Academic, at Digital Science. Dimensions Research Strategy launched 6 July 2026. Details at https://www.digital-science.