Stop Asking LLMs If Their Sources Are Real
You ask an AI for a list of sources. It gives you titles, authors, and DOIs. Everything looks perfect. Then you find out half of them do not exist.
The DOI leads to nothing. The paper was never written.
Your instinct is to ask the model: "Are you sure this is real?" The AI will say yes. It always says yes. You are asking a forger if his work is authentic.
An LLM does not use a database of papers. It predicts the next word in a sequence. To an AI, a citation is just a pattern. It knows a citation needs a name, a year, and ten digits. It mimics the shape of a real source without containing the truth.
Stop asking the model to verify its own work. It cannot do it for two reasons:
- It has no access to a live registry. It only generates plausible text.
- It has a bias toward confirming what it already said.
You must use external tools to verify information. Here is a three-step filter for technical writing:
Existence. Check the DOI against an API like Crossref. If the API returns a 404 error, the source is fake. An invented DOI will never resolve.
Credibility. A source can exist but still be worthless. Check if the journal or conference is recognized. A valid DOI does not guarantee quality.
Fidelity. Does the paper actually support your claim? You must read the content. Do not assume the abstract covers every detail. A real citation used for a false claim is still a lie.
This applies to more than just academic papers. If an AI cites a JIRA ticket, a CVE number, or a code commit, you must verify it against the actual system.
The biggest mistake in AI architecture is trusting the model to self-verify. Verification must be a separate step. It must connect to an external source of truth before the user sees the result.
Do not try to make models more honest. They are text generators, and they will always hallucinate. Instead, stop taking them at their word.
A citation you cannot verify against a registry is not a citation. It is a guess in a lab coat.
Source: https://dev.to/ohugonnot/stop-asking-the-llm-whether-its-source-is-real-2oaa
Optional learning community: https://t.me/GyaanSetuAi