𝗪𝗵𝘆 𝗠𝘆 𝗥𝗔𝗚 𝗕𝗼𝘁 𝗟𝗶𝗲𝗱 𝗔𝗻𝗱 𝗛𝗼𝘄 𝗜 𝗙𝗶𝘅𝗲𝗱 𝗜𝘁

𝗪𝗵𝘆 𝗠𝘆 𝗥𝗔𝗚 𝗕𝗼𝘁 𝗟𝗶𝗲𝗱 𝗔𝗻𝗱 𝗛𝗼𝘄 𝗜 𝗙𝗶𝘅𝗲𝗱 𝗜𝘁

I built a bot for internal documents. I used a standard RAG setup. It failed.

The bot hallucinated. It gave wrong numbers. It split long guides into small pieces. It found the wrong pages.

I fixed it with two methods.

• Parent-child chunking. I used small chunks for search. I sent the full section to the LLM. This gave the LLM full context.
• Hybrid search. I combined vector search with keyword matching. This found exact terms.

I added a reranking step. It filtered the top 3 results. I upgraded to GPT-4.

The results improved. The bot stopped lying.

RAG is not plug and play. It is a system design problem. How you slice and retrieve context matters most.

Tips for your build:

• Build an evaluation set.
• Monitor retrieval quality.
• Use reranking for better accuracy.

Source: https://dev.to/__c1b9e06dc90a7e0a676b/why-my-first-rag-system-hallucinated-and-how-i-fixed-it-cha