What if every thesis came with a reference check?

Nina Ghannam presents the second in her series of articles: Lessons from a Library in Lebanon
For most graduate students, the thesis is the last thing they submit before their research enters the scholarly record.
A lot of people look at it before then. The advisor and the committee read it closely, and in many institutions, mine included, librarians check the formatting. In my job at the thesis desk, I check whether the students followed the correct reference style. I don’t check whether those references are real or whether the papers behind them are still trustworthy.
I started thinking about this after a student emailed me last semester, a while after he had submitted his final thesis. He had found what he called an error in his references and wanted to replace the file. Since then, I have kept wondering how many similar mistakes nobody notices. So here is a thought experiment. Suppose every thesis came with a reference check that asked two things about each source: does it exist, and can it still be trusted?
Librarians have always cared about both. What worries me is how much easier it has become for a reference list to fail on both counts.
Does the source exist?
A few years ago, this question would have sounded silly. Students found a source, read it, and added it to their list. Today, a chatbot can produce a reference to a paper it has never seen, and sometimes to a paper that was never written.
These tools don’t look sources up the way a librarian would. They work by guessing which words are likely to come next, and they have seen so many citations that they know exactly what one looks like: a name, a year, a title, a journal. If no real paper fits, they can simply invent one, and it looks every bit as convincing. A study by William Walters and Esther Isabelle Wilder, published in Scientific Reports in 2023, showed how often this happens. When they asked ChatGPT for short literature reviews and checked the references, more than half of those produced by GPT-3.5 were invented. GPT-4 did better, but nearly one in five of its references still did not exist, and many of the real ones had wrong details.
These invented references are no longer only a classroom problem. In May 2026, a team led by Maxim Topaz at Columbia University published an audit in The Lancet of 2.5 million biomedical papers from the PubMed Central Open Access collection. In 2023, one paper in 2,828 contained at least one fabricated reference. By 2025, it was one in 458; in the first weeks of 2026, it was one in 277. When the team ran the audit, publishers had taken action on almost none of the papers affected.
Many students don’t know this, though AI is clearly on their minds. Since the AI declaration became part of our thesis guide, every student who comes to the thesis desk asks me about it, some with fear and others with confidence. Students everywhere now work with tools that make some of these choices for them. A reference manager formats the references a student gives it, while an AI assistant can produce a complete reference in seconds, whether or not the source exists. When everything looks neat, it is easy to accept the result without asking where it came from. If a tool formats a reference perfectly, it is natural to assume it also found it.
In principle, checking that a source exists is not hard. Each reference can be matched to a DOI or a record in a library catalog, and the details compared with the original. Free tools will do much of that work. But plenty of what students cite has neither. A history student might rely on letters kept in a family archive or drawer, or on a newspaper from the 1950s that has never been digitized. If a search turns up nothing, that tells us very little on its own. Librarians have always helped track down sources like these, and it usually starts with knowing which collection to ask.
There is another problem as well. A reference can be real and correct in every detail, and the source can still fail to say what the student claims it says. A student might cite a study as evidence for a point the study never made, or even one it argued against. Software can flag possible problems, but it can’t reliably tell whether a source supports an argument. Someone has to read the work, and in a thesis that someone is usually the advisor or a committee member who knows the field.
Can the source still be trusted?
This second question is harder, because here the source is real. A paper can be published, cited for years, and then retracted when the journal decides its findings can no longer be relied on. Sometimes the cause is an honest mistake; sometimes it is fabricated data. A retraction is not always a sign of misconduct, but it does mean the findings should no longer be treated as reliable evidence or as a trustworthy source of information. There can be good reasons to cite a retracted paper, for example, when the retraction itself is part of the research. What matters is knowing that it was retracted, and why. Retractions are not rare. According to a Nature analysis, more than 10,000 papers were retracted in 2023. The paper itself usually stays online with a notice attached, but copies saved in reference libraries and on laptops carry no notice, so a student can cite one years later without knowing.
The problem can be measured, at least once it reaches published research. The Research Integrity Risk Index (RI²), developed by AUB University Librarian Dr. Lokman Meho, gives institutions a way to see it. One of RI²’s four indicators, the C-Rate, measures the share of an institution’s recent publications that cite papers retracted more than 12 months earlier. RI² places each institution in a risk tier, from low risk up to the highest level of concern. When the C-Rate was added, about one in five of the 2,000 institutions in the index moved to a different tier, and 42 of them ended up in the two highest-risk tiers for that reason alone. The advice in the same report caught my attention because it is close to what I would tell a student: set your reference manager to flag retracted papers and look for retractions while you are still doing the research, not only at the very end. Because RI² works with published, indexed articles, it shows where things stand once research is in print. Theses come earlier. They are one place where the same question could be asked before anything is published.
Reference managers can help; Zotero and EndNote both flag retracted papers using data from the Retraction Watch database. The alerts only work for items with a DOI or a PubMed identifier, and the database is incomplete, so a clean result is no guarantee.
Who should do the checking?
This is where the idea gets difficult. Libraries that review theses often have small teams, and most submissions land in the same few weeks each semester. We can’t verify every reference in every thesis, and choosing sources is not our call anyway. That is for the students, with their advisor and committee.
Students can do the check themselves, though, and the earlier the better. AI tools are also getting better at this. Students can ask the same tools (or a competitor tool) that produced their references to check whether any source in their list, however they found it, has been retracted or does not exist. These tools can still be wrong, so anything doubtful should be confirmed against the DOI or the Retraction Watch database. By the time a thesis reaches the final review, students are racing a deadline, and there is little room to go back through their sources.
A short exercise in a research methods course, or a checklist students keep as they write, would make more sense than one more step at submission. Students could then run it once more just before submitting, when there is still time to fix anything they have missed. The library’s part would be to teach the check and to help when something is hard to trace. And we would keep telling students what we already tell them: if a tool gave you the reference, it is still your reference and your responsibility. An advisor could also ask about the check as casually as they ask whether the spelling has been looked at.
There is also what happens after submission. Once a thesis goes into an institutional repository, it can stay available for decades. Other researchers find it, read it, and follow its references, so a mistake in the bibliography can travel much further than the student who made it. That seems reason enough to help students catch what they can before the thesis becomes part of the university’s collection.
When should students learn to check?
I don’t know if a check like this would ever become a standard practice. What I do know is that AI has made it easy to produce a reference to something that doesn’t exist, and that RI² now lets universities see how often their researchers cite work that has been retracted.
These are different problems. One is a reference to something that was never there; the other is a reference to something real that should no longer be relied on. But both end the same way, with a reference that looks perfectly sound on the page and quietly carries a problem into the scholarly record. Students need to learn to check their sources, and the best time to learn is while they are still writing.
If I could choose when students first think about these questions, it would be in their first semester, with a librarian beside them and plenty of time to fix whatever turns up. A check learned that early is one they can carry into their own research long after the thesis is finished. I would be glad to hear from anyone who has tried something like it.

Nina Ghannam is a Reference and Instruction Assistant Librarian at the American University of Beirut
References
Albert Einstein College of Medicine Library. (n.d.). Retractions flagged in EndNote. https://library.einsteinmed.edu/blog/Retractions-flagged-in-EndNote
Ghannam, Nina. (2026, October 29). Lessons from a library in Lebanon. Research Information. https://www.researchinformation.info/analysis-opinion/lessons-from-a-library-in-lebanon/
Meho, L. I. (2025). Gaming the metrics: bibliometric anomalies in global university rankings and the research integrity risk index (RI2). Scientometrics 130 (11), 6683-6726. https://doi.org/10.1007/s11192-025-05480-2
Meho, L. I. (2026). Research Integrity Risk Index (RI²) [Interim 2026 edition]. https://sites.aub.edu.lb/lmeho/ri2/
Meho, L. I. (2026). What changed when citation exposure to retracted research was added to the risk index? [Special report]. Research Integrity Risk Index. https://sites.aub.edu.lb/lmeho/ri2/c-rate/
Topaz, M., Roguin, N., Gupta, P., Zhang, Z., & Peltonen, L.-M. (2026). Fabricated citations: An audit across 2·5 million biomedical papers. The Lancet, 407(10541), 1779–1781. https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(26)00603-3/fulltext
Van Noorden, R. (2023). More than 10,000 research papers were retracted in 2023: A new record. Nature. https://doi.org/10.1038/d41586-023-03974-8
Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13, Article 14045. https://doi.org/10.1038/s41598-023-41032-5
Zotero. (2019). Retracted item notifications with Retraction Watch integration. Zotero Blog. https://www.zotero.org/blog/retracted-item-notifications/
