𝗜 𝗕𝘂𝗶𝗹𝘁 𝗥𝗔𝗚 𝗙𝗿𝗼𝗺 𝗦𝗰𝗿𝗮𝘁𝗰𝗵 𝗶𝗻 𝗣𝘆𝘁𝗵𝗼𝗻 𝘁𝗼 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝗜𝘁

I used LangChain in production for six months. I could not explain how it worked. I did not know why I chose specific metrics or how text became vectors. The library hid the logic.

To fix this, I deleted the framework. I wrote a RAG pipeline from scratch using 500 lines of plain Python.

Here is what I learned from building the stack manually.

The Problem with Black Boxes When you use high-level libraries, you lose control. I saw models hallucinate facts or give wrong citations. I could not tell if the error lived in the chunker, the embedding model, or the prompt.

When you build it yourself, every layer is inspectable. You can print the exact chunks sent to the LLM. You can see exactly where a sentence breaks.

The Five Layers of RAG RAG is not one algorithm. It is five different processes stacked together:

  • Chunking: Deciding how to split text.
  • Embedding: Turning text into math.
  • Retrieval: Finding the right pieces.
  • Prompt Construction: Telling the model how to behave.
  • Generation: Getting the final answer.

Lessons from the Build

  1. Chunking is the most important step Most tutorials skip this. If you do not use overlap, you lose context at the boundaries. I used a sliding window with character-level overlap. This ensures the model sees the connection between two chunks.

  2. Distance metrics matter I spent hours debugging bad search results. The issue was not the data. It was the metric. ChromaDB defaults to L2 distance. For semantic search, you need Cosine similarity. One line of code changed everything.

  3. Prompts need constraints An LLM is a completer, not an oracle. If you ask a vague question, it will invent an answer. I learned to use a strict refusal template. I told the model: "If the context does not contain the answer, say you do not know." This dropped hallucinations from 40% to 5%.

  4. Batch your requests Sending one HTTP request per chunk is slow. Sending them in batches is much faster. It allows the local model to pipeline the work.

  5. Test from the bottom up Do not write tests at the end. Test your chunker first. Then test your embedder. Then test your store. If you test last, you test the bugs instead of the logic.

If you feel like you do not truly understand your AI stack, build it yourself. The code is not the goal. The thinking is the goal.

Source: https://dev.to/avinash_zala_1c6f5e7c4af9/i-built-rag-from-scratch-in-python-to-understand-it-heres-what-i-learned-33kf

Optional learning community: https://t.me/GyaanSetuAi