My Agent Shipped 3 PRs In An Evening. 40% Of My Messages Were Corrections.
My AI-driven coding agent pushed three pull requests in a single evening, but 40 % of the 30 messages I sent were corrections.
The session produced an MCP client, an Azure AI Agent, and an M365 Copilot Agent. Automated checks cleared all three PRs, and I never edited a line of code. Yet the transcript tells a different story: out of 710 total messages, I typed 30, and 12 of those steered the agent back on track. The “steering rate” – the share of my messages that were corrections – sits at 40 %.
How the pipeline was wired
- Claude drafted a high-level implementation plan.
- DeepSeek V4-Flash acted as the orchestrator, reviewing the plan.
- Codex generated the actual code.
- The orchestrator inspected the code and opened the pull requests.
The orchestrator’s intended role was purely connective – it should resolve conflicts between components, not write code itself. In practice, the agent produced 3,500 lines of code across the three PRs in roughly 40 minutes, but it also slipped on two recurring error classes.
The two error families
- Workflow violations – the orchestrator occasionally took over the coding step, ignoring its “glue” role and writing implementation details itself.
- Context-retrieval failures – despite explicit instructions, the agent selected the wrong SDK or version. The correct information sat in the prompt context, but the model failed to surface it at the right moment.
These are not gaps in reasoning ability; they are engineering bugs in how the workflow is constrained. Even a more capable language model would still need a hard, unmissable rule that pins the orchestrator to its non-coding duties and forces the correct SDK selection.
What I changed to tame the agent
I stopped assuming the system would infer its role from the step list. I added a direct statement: “You are an orchestrator. You do not implement.” It took five corrective messages for the instruction to stick, after which the agent respected the boundary.
I also tightened the context-retrieval logic. When the wrong tool appeared, I treated it as a bug in the retrieval pipeline rather than a hallucination, and I rewrote the prompt that feeds SDK details to make the correct version impossible to miss.
Practical takeaways for AI-augmented development
- Count your own messages. A high volume of accepted PRs can mask a broken process. Your correction count is a leading indicator of where the harness is leaking.
- State the role explicitly. Agents do not extrapolate identity from a checklist; they need a clear, pinned instruction about who they are and what they may do.
- Treat tool-choice errors as engineering bugs. If the agent ignores a specified SDK, the fault lies in the context-delivery mechanism, not in the model’s “knowledge.”
- Convert mistakes into reusable skills. I let the agent generate a validation routine from its own errors, turning a failure into a future safeguard.