You wake up to five critical bug reports on a Monday morning. Your review monitoring tool has done its job. It caught every crash report, every angry one-star review, every "app freezes when I tap save." You know exactly what is broken. What you don't know is where to look.
That was the wall I hit after building my first pipeline. It monitored app reviews and incoming crash logs without trouble, sorting each piece of feedback into tidy buckets: bugs, crashes, or feature requests. The dashboard looked healthy. The actual debugging process was not.
Knowing a bug exists is only the first inch of the mile. I still had to open the IDE, grep through modules, cross-reference stack traces against the current codebase, and reconstruct the failure path in my head. When the tickets are piling up and the coffee is still hot, that manual archaeology burns time you do not have. I needed the pipeline to do more than flag problems. I needed it to investigate them.
So I rebuilt the system around a single goal: take a raw bug report and return a validated diagnosis. Not a paragraph of LLM musings. A structured finding that names the file, points to the line, estimates the risk, and suggests a fix. Here is how it came together.
Why Structure Beats a Chat Log
I built the investigating agent with PydanticAI. The reason was simple. When you ask a language model to reason about code, its default output is a friendly stream of text. That might help a human reader, but it is useless to a downstream script. I needed a machine-readable contract.
The agent returns a validated data model with four specific fields: the root cause, the affected files, the proposed changes, and an assessment of complexity and risk. If the model is missing a field or hallucinates a filepath, validation fails and I catch it immediately. That rigor keeps the pipeline honest.
To do the actual detective work, the agent gets four read-only tools and nothing else. It can search code via grep, read specific line ranges from a file, list directory contents, and locate symbols like classes or functions. Read-only is the important part. I did not want an agent with write access wandering around my repository at 2 a.m. First understand, then edit.
The Repo Map: Context Before Tools
The first version of the agent was accurate but wildly expensive. It burned through tokens like a tourist walking in circles. The model would call list-dir, then grep, then read a file, then list-dir again, slowly assembling a mental model of the project structure one expensive token at a time.
The fix was to generate a compact repo map before the agent ever starts. This map is a distilled overview of the repository: key files, their primary functions or classes, and how the major modules connect. Think of it as handing the agent a GPS instead of asking it to discover the roads by trial and error.
With that map in its context window, the agent does not waste calls figuring out that src/utils/parser.ts exists. It already knows the terrain. It heads straight to the ridge where the smoke is rising. That single change cut out the wandering phase entirely.
The Tool Funnel: Forcing a Conclusion
Even with a map, the agent could dither. It would find a suspicious file, then second-guess itself, then search again, then read another file, caught in an endless loop of just-one-more-check. I needed a way to force momentum.
I implemented a three-phase tool funnel that restricts what the agent can do as it progresses.
Phase one is exploration. The agent has full access to all four tools. It can search, browse, and read whatever it needs to replicate the bug in its reasoning.
Phase two is deep-dive. Once the agent has identified the likely fault lines, it loses the discovery tools. It can only read files. No more grepping, no more directory listings. At this stage it must study the code it has already found and build its chain of evidence.
Phase three is output. All tools are locked away. The agent cannot query the codebase anymore. It has to sit down and write the report. This prevents the endless "let me check one more thing" spiral.
That funnel dropped the average number of tool calls from over forty per analysis down to roughly ten. The agent got faster, cheaper, and paradoxically more confident because it had to commit to a conclusion.
Keeping the Backend Swappable
میں سسٹم کو کسی ایک ماڈل فراہم کنندہ کے ساتھ ہارڈ کوڈ نہیں کرنا چاہتا تھا۔ میں کام کی نوعیت کے لحاظ سے مختلف انجن استعمال کرتا ہوں۔ کبھی Claude Code، کبھی Grok Build، اور کبھی وہ جو اس وقت سب سے سستا ہو۔ بنیادی منطق کو فراہم کنندہ سے آزاد رکھنے کے لیے، میں نے کام کو دو مراحل میں تقسیم کر دیا ہے۔
پہلا مرحلہ 'تلاش' (exploration) ہے۔ کوڈنگ ایجنٹ، جو کوئی بھی قابل ماڈل ہو سکتا ہے، ریپو میپ (repo map) کو پڑھتا ہے، ٹولز کا استعمال کرتا ہے، اور ایک خام مارک ڈاؤن رپورٹ تیار کرتا ہے۔ یہ سوچنے والا مہنگا مرحلہ ہے۔
دوسرا مرحلہ 'ساخت سازی' (structuring) ہے۔ ایک سستا اور تیز LLM اس مارک ڈاؤن کو لیتا ہے اور اسے سخت Pydantic ماڈل میں دوبارہ فارمیٹ کرتا ہے۔ اس مرحلے میں تقریباً کسی استدلال (reasoning) کی ضرورت نہیں ہوتی۔ یہ محض ڈیٹا نکالنے اور فارمیٹ کرنے کا کام ہے، اس لیے یہ ہلکے پھلکے ہارڈ ویئر پر بھی چل سکتا ہے۔
چونکہ ان کے درمیان فرق واضح ہے، اس لیے میں ویلیڈیشن لاجک کو چھیڑے بغیر بیک اینڈ تبدیل کر سکتا ہوں۔ مارک ڈاؤن رپورٹ، 'تلاش کرنے والے دماغ' اور اس 'ساخت شدہ آؤٹ پٹ' کے درمیان ایک یونیورسل اڈاپٹر کے طور پر کام کرتی ہے جسے میں اصل میں استعمال کرتا ہوں۔
حقیقت میں کیا کام آیا
اس سیٹ اپ نے آنے والے مسائل کو سنبھالنے کے میرے طریقے کو بدل دیا ہے۔ کلاسیفیکیشن لیئر اب بھی بگ (bug) کو فیچر کی درخواستوں سے الگ کرتی ہے، لیکن اب تجزیاتی لیئر فوراً اس کے بعد کام شروع کر دیتی ہے۔ جب تک میں اپنا ایڈیٹر کھولتا ہوں، میرے پاس فائل پاتھ، لائن رینج اور ایک مجوزہ تبدیلی پہلے سے موجود ہوتی ہے۔ میں اب بھی ہر چیز کا خود جائزہ لیتا ہوں۔ یہ مدد ہے، آٹو پائلٹ نہیں۔ لیکن سیاق و سباق جمع کرنے کا وہ عمل جو پہلے