A new WhatsApp-based RAG (retrieval-augmented generation) assistant stopped hallucinating by demanding that every answer be backed by a JSON-encoded citation. When the model could not point to a specific chunk, it returned “I don’t have enough information to answer that,” turning a confident liar into a trustworthy “I don’t know” system.

Why trust matters in RAG

Most tutorials on RAG obsess over retrieval—choosing embeddings, slicing documents into chunks, re-ranking results. They skip what happens after the relevant text is found. Users lose faith when the assistant offers a definitive answer that the retrieved context does not actually support.

The confidence gap

In a prototype built with PostgreSQL and the pgvector extension, the retrieval pipeline was straightforward. The real challenge appeared in production: the language model sounded sure even when the retrieved snippet lacked the needed information. A demo could hide the problem, but real users exposed the gap between sounding confident and being correct.

Enforcing citations with JSON

Instead of coaxing the model with prompts like “answer only if you have context,” the author changed the output format. The system now requires a JSON object where each claim includes a reference to the exact chunk that supports it. If the model cannot attach a citation, the response is rejected and the user sees a clear “I don’t know” message.

The enforcement moved from natural-language instructions to schema validation. The model still generates text, but surrounding code checks that the JSON matches the required structure before it reaches the user.

What changed

  • Chunking became conservative. Vague or overly broad chunks now generate claims without citations, triggering the “I don’t know” fallback.
  • System prompt shrank. The heavy-handed prompt that tried to police the model’s behavior was replaced by a short instruction set, letting the schema do the heavy lifting.
  • Failures are visible. When retrieval returns irrelevant material, the assistant no longer masks the error with a confident but wrong answer; it openly admits uncertainty.

Takeaway: For RAG assistants, guaranteeing that every claim is traceable to a retrieved source builds user trust more reliably than polishing the retrieval step alone. By turning missing citations into a visible “I don’t know,” developers let the system admit its limits instead of fabricating answers.