Chat-based AI "not enough for scientific research"

Some 57% of AI users are not fully satisfied with the tool they use,  according to a new study by Iris.ai of over 500 corporate research professionals, aimed at assessing their use of AI in their research and the challenges they face in doing so.

The research found that the most prominent tool for assisting in research is currently ChatGPT,  employed by 89% of AI users, suggesting that chat functions with AI tools may not be solely suitable for aiding scientific researchers. Less than a quarter of respondents use more complex solutions to reduce research timeframes: 23% using AI to summarise individual papers and optimise searches, and 21% using AI to extract knowledge from bodies of research. Many of these functions are not found in general-purpose AI tools.

The most common issues for with using AI for scientific research were:

• Inaccuracy (59%).
• Misinformation (46%).
• Lack of citations (42%).

With non-domain-specific models limited to simple tasks and lacking the ability to leverage external data sources, the authors of the report say there is a clear demand for a more suitable AI assistant for scientific research. Conversational abilities need to be paired with sophisticated visualisations, such as knowledge graphs, and the ability to measure the factuality of outputs. Allowing AI to provide detailed citations and explain the steps it has taken to reach a particular response would allow scientific researchers to use their own training to efficiently fact-check its responses. 

Victor Botev, CTO and co-founder of Iris.ai, said: “Interacting with a user interface exclusively through a chat function is seen as something of the past. For AI to be truly effective, in science and beyond, we need to prioritise different forms of engagement. Mustafa Suleyman has said the next stage of generative AI is interactive AI, and I agree. We’re already seeing an appetite for this from scientific researchers.

“It's clear from our study that, whilst researchers recognise the potential of AI, they're still looking for a tool that truly meets their needs. Our research team has developed our own metrics and algorithms to provide accurate, reliable, and comprehensive solutions that not only improve the research process but also enhance trust in AI capabilities. It's an ongoing journey, but we're excited to be at the forefront of this transformation."

When researchers were asked about the features that would make them trust or use generative AI more, the top responses were citations on origin (49%) and quality (46%) of data, metrics on correctness and uncertainty (45%), and summarisation of individual scientific research papers (45%).

The study was carried out on a sample of 500 scientific researchers across a variety of disciplines (including industrial engineering, chemistry, oil & gas producers, alternative energy, and food production) at organisations from <10 to >5000 employees.