𝗪𝗵𝘆 𝗠𝘆 𝗥𝗔𝗚 𝗔𝗽𝗽 𝗛𝗮𝗹𝗹𝘂𝗰𝗶𝗻𝗮𝘁𝗲𝗱 𝗔𝗻𝗱 𝗛𝗼𝘄 𝗜 𝗙𝗶𝘅𝗲𝗱 𝗜𝘁
I once demoed my RAG support bot to a colleague.
The bot told her our refund policy was 30 days.
Our actual policy is 14 days.
The bot did not say it was unsure. It made up an answer with total confidence.
RAG should stop hallucinations. My setup only moved them around.
I fixed this by following five steps.
- Fix your chunking strategy
I used to split documents by character counts. This was a mistake.
One chunk often mixed different topics like shipping and returns. The model blended these into wrong answers.
The fix: I switched to semantic chunking. I now split text by headings and paragraphs.
- Stop trusting similarity scores alone
My retriever pulled the top three chunks based on cosine similarity.
Similar is not the same as relevant. A chunk can look similar to a question but contain no answer.
The fix: I added a reranking step with a cross-encoder. This helped me see when the system had no real data.
- Give the model permission to fail
My old prompt only said: Use the context to answer the question.
If the context lacked the answer, the model filled the gaps with guesses.
The fix: I added a direct instruction. If the answer is not in the context, say you do not know. This alone stopped most hallucinations.
- Set a retrieval threshold
Sometimes the model used general knowledge when retrieval failed. I was hoping the prompt would work, but hope is not a strategy.
The fix: I added a score cutoff. If the retrieval score is too low, the system returns a fallback response. No context means no answer.
- Test for failure, not success
I used to test only easy questions. I ignored ambiguous queries and missing information.
The fix: I built an evaluation set of trap questions. These are cases where the answer is missing. Running these tests regularly shows you exactly where your system breaks.
RAG does not eliminate hallucinations. It makes them controllable.
My bot still does not know every detail. But now, it admits when it is unsure. That makes it usable.
Source: https://dev.to/pallavi_sharma_10c1a6f1da/why-my-rag-app-kept-hallucinating-and-how-i-fixed-it-3i10
Optional learning community: https://t.me/GyaanSetuAi