๐๐ฒ๐๐๐ผ๐ป๐ ๐ณ๐ฟ๐ผ๐บ ๐ฎ ๐ญ๐ฌ๐ต-๐ฎ๐ด๐ฒ๐ป๐ ๐ฐ๐ผ๐ฑ๐ฒ ๐ฎ๐๐ฑ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐ณ๐น๐ผ๐
I spent 9.3M tokens on a code audit so you do not have to.
I used a swarm of AI agents to find bugs in a 5,000-line codebase. The system used mappers, finder lenses, deduplication, and adversarial verification.
It worked. I got 32 verified findings and a clean top-10 list. But it cost $46 in API fees. Most of that money was wasted.
Here is what went wrong:
โข Verification was too expensive. 86 of the 109 agents were verifiers. They only caught 2 errors. I paid to re-read code 86 times for a 6% success rate.
โข Mapping was redundant. The finders re-read the code anyway. The map phase was an extra tax.
โข Finders overlapped. There was 30% overlap between the 8 lenses.
โข Formatting wasted money. Using pretty-print JSON bloated every prompt by 40%.
โข Cache reads were high. Every agent re-reads the same files from scratch.
How to fix your AI workflows:
โข Rank before you verify. Find findings, deduplicate them, and rank them. Only verify the top 15. This uses 70% fewer agents.
โข Match paranoia to stakes. Use one verifier for internal audits. Use a full panel only for findings that require real action.
โข Batch verification by file. If 34 findings live in 10 files, make one verifier read the file once. Do not make ten verifiers read the same file.
โข Skip mappers for small repos. If the code is under 10,000 lines, one agent can read it all.
โข Limit your lenses. Use six lenses max. Give each lens clear boundaries so they do not repeat work.
โข Compact your JSON. Do not use extra spaces or new lines in your JSON strings.
โข Use cheaper models for chores. Use frontier models for logic. Use cheap models for deduplication and evidence checking.
โข Set a token budget. Have your orchestrator check the remaining budget before starting new tasks.
The lesson is simple: Fan out to find, but converge before you verify. Breadth is for discovery. Rigor is for the survivors.
Source: https://dev.to/ayoubzulfiqar/lessons-from-a-109-agent-code-audit-workflow-4a5m