Timu nyingi hujenga mfumo wao wa kwanza wa upatikanaji (retrieval) kwa njia ileile: kukata kila hati katika vipande vya tokeni 512 vilivyofungwa, kuviweka kwenye kanzidata ya vector, na kutumaini kuwa modeli ya embedding itafanya kazi ngumu. Tumaini hilo linakusaidia tu wakati wa onyesho (demo). Halidumu unapokutana na watumiaji halisi.
Katika mazingira ya uzalishaji (production), mkataba wa kisheria unaharibika unapojitenga kipengele cha dhima (liability clause) kutoka kwa vighairi vyake. Maelezo ya API yanakuwa hayana faida wakati sampuli ya kodi inapotengana na saini yake ya kazi (function signature). Mfululizo wa mazungumzo ya huduma kwa wateja unakuwa kelele unapotoa malalamiko moja pekee kutoka kwenye historia yake ya mazungumzo. Tatizo mara nyingi si modeli ya lugha iliyo mwisho wa mchakato (pipeline). Tatizo ni kile unachokiingiza.
Tulijifunza hili kwa njia ngumu. Tabaka letu la kwanza la upatikanaji lilionekana kuwa la kawaida lakini lilifanya kazi bila uthabiti. Hivyo tulijenga upya kuzingatia wazo rahisi: chukulia upatikanaji kama miundombinu inayopimika, si uchawi. Hapa kuna kile kilichobadilika hasa, na jinsi tulivyoongeza uwezo wa upatikanaji (recall) hadi asilimia 95 huku tukipunguza ucheleweshaji (latency) wa asilimia 95 kutoka ms 850 hadi ms 320.
Mtego wa Vipande Vilivyofungwa (Fixed-Chunk Trap)
Idadi sawa ya tokeni ni rahisi kuandika kodi na rahisi kuelezea. Urahisi huo unaficha ukweli wa msingi: hati zina muundo. Unapopuuza muundo huo, unaharibu ishara (signal).
Fikiria mkataba mkuu wa huduma wa kurasa kumi. Kipande cha tokeni 512 kilichofungwa kitatua katikati ya wajibu, kikitenganisha kipengele na jedwali la ukomo (cap table) linalokizuia. Hatua ya upatikanaji kisha inarudisha nusu ya wazo. Kigenereta (generator) kinatunga sehemu iliyobaki (hallucinates). Katika maelezo ya API, kipande kikubwa mno kinachafua embedding kwa vichwa vya habari vya kawaida (boilerplate), kikificha njia (method) mahususi anayohitaji msanidi programu. Katika tiketi za msaada, dirisha lililofungwa linachukulia mazungumzo kama mkusanyiko wa sentensi, likiondoa mwingiliano unaofichua kile kilichofeli hasa.
Tulimaacha kutendea ukubwa wa kipande kama hyperparameter tunayokisia. Tulianza kuuchukulia kama zoezi la ulinganishaji (mapping) kati ya aina ya hati na usanifu wa habari (information architecture) iliyomo ndani yake.
Linganisha Mgawanyo Wako wa Vipande na Data
Suluhisho si ukubwa mmoja kamili wa kipande. Suluhisho ni mbinu tatu tofauti zilizorekebishwa kulingana na aina tatu tofauti za data.
Hati za kisheria sasa hupitia mgawanyo wa mfuatano (recursive splitting). Algorithim kwanza hutafuta mipaka ya asili mikubwa zaidi—sehemu, kisha sehemu ndogo, kisha vipengele vilivyopewa namba—na kutumia vipande vidogo tu inapohitajika. Hii inafanya kipengele cha kusitisha (termination clause) kibaki kikiambatana na masharti yake ya kuendelea. Hatua ya upatikanaji huona vitengo kamili vya kimantiki, jambo ambalo hupunguza sana mshawasha wa modeli kutunga vighairi vilivyokosekana.
Maelezo ya API na kodi hupata mgawanyo unaozingatia muundo (structure-aware chunking). Vichwa vya habari vya Markdown, code fences, na majedwali ya vigezo (parameter tables) huchambuliwa kama vitengo vya msingi (atomic units). Hatugawanyi ndani ya kodi (code block). Tunabakiza docstrings karibu na saini (signatures) zake. Matokeo yake ni kwamba hoja (query) ya njia ya darasa (class method) mahususi inarudisha muktadha kamili anaohitaji msanidi programu: maelezo, vigezo vilivyowekwa aina (typed parameters), na mfano unaofanya kazi.
Data za msaada na mazungumzo hutumia mgawanyo wa kimaana (semantic chunking). Badala ya kuhesabu tokeni, tunatafuta mabadiliko katika mada au nia. Ikiwa mteja anaelezea hitilafu (bug) katika ujumbe wa tatu na kuweka stack trace katika ujumbe wa saba, tunagawanya kwa maana, si kwa namba ya ujumbe. Tabaka la upatikanaji kisha linarudisha mfululizo kamili wa tatizo badala ya sentensi iliyotengwa (orphaned sentence).
Kwa Nini Utafutaji wa Vector pekee Unashindwa
Hata vipande kamili hupotea katika utafutaji wa vector pekee. Dense embeddings ni bora katika kunasa maana na uafanani wa maneno (synonymy), lakini zinajulikana kwa kutokuwa na usahihi wa maneno kamili (exact strings). Ikiwa mhandisi anatafuta kodi ya hitilafu mah
Users do not write ideal search queries. They paste truncated log lines. They type “it’s broken.” They use jargon your documentation never adopted. If you trust the raw query, you are trusting noise.
We now expand every incoming query into three to five variations before sending them to the retrieval layer. One variation might be a direct paraphrase. Another might be a hypothetical ideal document title. A third strips away conversational filler and isolates technical keywords. Each variant gets embedded and searched. We then deduplicate and merge the candidate pools.
This is not free. Those extra embedding calls cost money and add a few milliseconds. But the effect on recall was dramatic: we moved from 78 percent to 96 percent by expanding queries before retrieval. Because better retrieval shrinks the generation window and grounds the model in correct context, we ended up saving money downstream. A slightly more expensive retrieval step is cheaper than a long, hallucinated generation step.
Stop Guessing. Start Searching.
Once we had the right chunking, hybrid retrieval, and query expansion in place, we still faced a combinatorial mess. Chunk size, chunk overlap, top-k retrieval depth, reranker cutoffs, and fusion weights all interact. A manual grid search would have taken weeks and still left us with a local maximum.
We switched to Bayesian optimization to explore the space. Rather than exhaustively testing every combination, the search algorithm maintains a belief over which configurations are likely to perform well and progressively narrows in on promising regions.
The output is not a single perfect setting. It is a Pareto frontier of choices. On one end, we have a lean configuration optimized for our high-throughput API support endpoint: fast inference, moderate recall, and the lowest possible latency. On the other end, we have an aggressive configuration for legal review: deeper retrieval, heavier reranking, and tighter overlap, trading milliseconds for thoroughness. Because the frontier is explicit, we can pick the right point for the product instead of pretending one size fits all.
What the Numbers Actually Look Like
These changes moved the system from a brittle prototype to a measured production pipeline.
Recall at ten improved from 78 percent to 95 percent. That means when the correct answer exists in our corpus, we find it nineteen times out of twenty.
Latency at the 95th percentile dropped from 850 ms to 320 ms. The hybrid stack sounds heavier on paper, but smarter indexing, smaller rerankers, and the ability to serve aggressive chunks only when needed made the whole system faster.
Hallucination rate—tracked by human annotators on a held-out golden dataset—fell from 12 percent to 3 percent. When the model receives complete, relevant context, it stops inventing facts.
Cost per query dropped from $0.008 to $0.005. Better retrieval means shorter, more focused LLM prompts and fewer recovery attempts. The extra embedding spend on query expansion is dwarfed by the savings in generation.
Build a Golden Dataset and Treat Retrieval Like Code
If you take one thing from this, it should be the discipline of measurement. We built a small golden dataset of real questions and verified answer locations. Before any change hits production, it runs against that dataset. Recall and latency are monitored in real time, not eyeballed in a notebook.
Retrieval is not a research demo. It is infrastructure. It deserves unit tests, regression benchmarks, and automated optimization just like the rest of your stack. Chunk by document structure, not by token superstition. Combine vector and keyword search with a reranker. Expand the queries your users actually write. Then let a search algorithm tune the knobs instead of your intuition.
The pipeline we described is not theoretical. You can read the original write-up here, and if you want to discuss retrieval engineering with a community that cares about this stuff, the GyaanSetu AI group is open.