Supporting credible scholarship with AI functionality
When thoughtfully considered, the functionality and features of an AI tool can be used to enable the research process in a way that supports responsible, credible research, writes Yoshiko Kakita
Researchers rely on credible evidence as a foundation to their work. Their rigorous research moves society forward, powering effective societal decision-making.
However, the rising use of AI as a research tool has the power to shift researchers away from trusted content and remove necessary barriers that protect the verification and transparency of the research process.
- Potential for hallucination and bias: Misinformation has become a larger problem with the rise of AI tools, partly from hallucinations or bias. Bias can be introduced during design, development, creation of algorithms or amplified through the data used during algorithm training. Hallucinations spread misinformation by generating false information and citations. Without human evaluation and verification, these put researchers and institutions at risk.
- Reduced verifiability and transparency: Depending on the features of an AI tool, the search mechanism as well as the sources used to generate the response may be unclear. This means the user cannot verify correctness or bias in results. This becomes even more concerning when AI tools are used in decisions for healthcare, law, policy or other fields that have real-world impacts.
- Threats to autonomy and privacy: Some AI tools may use input information to train their tools, meaning information users share may not be private. This can be even more concerning when users enter sensitive information. In addition, AI responses raise questions about responsibility for creation of knowledge. This can be especially tricky when AI responses cannot be traced back to a source.
Protecting the research process
The features and functions of AI tools can act as guardrails to protect against threats to the research ecosystem, promoting increased confidence in robust findings that can be used for future research. Note, this is not a substitute for AI literacy education, but rather an additional layer of checks.
When using AI tools for research, the following five features and functions are necessary to promoting sound research practices.
Claim-to-source accuracy
Evaluating claims is essential to determine the truthfulness of the statement. This starts with citations or links to the original source. An advanced research tool may provide extra guidance on the relationship of the original content source to the claim, for example information on how closely they match. This guidance helps the user decide to review the original source or move to the next relevant source. The goal with this feature is to surface the evidence and let the user be the judge.
Research-level context
Because research is an evolving process, evidence on some research topics can be conflicting. Without the scientific context, an AI tool may confidently claim a conclusion despite ongoing scientific debate. This can present a problem for researchers when trying to understand a topic. An AI tool for research may include features helping to place claims into the context of the wider-research conversation, providing background information to support the full spectrum of findings. This helps researchers to better understand the nuances in the response and within the scientific record.
Responsible development and maintenance
AI tools for research require responsible development and maintenance to help them support research objectives. Responsible development can start with a set of responsible AI principles that serve as a guide for tool creation and updates. For research purposes, principles of tool development should be centered around humans, focusing on transparency, privacy and user control.
Beyond responsible design principles, learn about other evaluations in place to keep the tool accountable. Continual algorithm evaluation is one such evaluation. This supports the tool staying consistent, true to original sources and meeting its original goals. Other independent checks, like a review board, may also be helpful in providing another layer of tool verification to keep research safeguarded.
Explainability and interpretability
AI tools, but particularly ones for research, are more useful when explainable and interpretable. Explainable means that a system can explain how it arrived at the answer, while interpretable means that a human can understand how the response was generated by understanding how the tool functions. These principles are important for providing transparency in tool responses and increasing confidence in the response and improving verifiability.
Privacy and security
When working with any AI tool, users should be clear about how their data is stored and used. Working in research adds another layer of importance. If researchers are working for an institution, they must abide by their privacy and security policies. Research funded by grants may have similar policies. Aside from legal compliance, researchers may wish to ensure their inputs remain confidential to keep their insights from being shared.
To support these efforts, users or their institutions can check with their provider of an AI research tool on their policies. Start with learning about the use of inputs into the AI research tools. Understand if data is used to train models, what data is retained, if it’s encrypted and how it’s stored. Finally, review any tool policies on privacy and security, making sure the tool is compliant with institution, funding or country level policies.
Making a tool decision
The potential of AI tools in the research space is real, but so are the risks. Evaluating tools for use, whether at an individual or institutional level, is paramount for responsible, transparent use with verifiable results. Tools like LeapSpace are built with these considerations in mind, helping researchers use AI while engaging in responsible research practices.
Yoshiko Kakita is VP, LeapSpace Product Management, at Elsevier