How I Made AI Stop Hallucinating on Our Codebase
AI coding tools fail on real production projects. They work well on new code. They fall apart on old code with history.
Our fintech project has three years of history. It has two React frontends, an admin panel, and a FastAPI backend. The database is complex with many tables and heavy joins.
We tried to use AI to move faster. It failed.
I asked the AI to create a contacts table. It created new columns for names and emails. It did not realize we already had these in our users table. It duplicated data instead of using a foreign key.
The AI was not stupid. It had no context. It made a decision with incomplete information.
I stopped asking how to get better code. I started asking what context the AI needs to make good decisions.
We built a structured workflow. AI is only as good as the context you provide. We made that context explicit.
Here is our setup:
- ADR Directory: We created a folder for Architecture Decision Records. These files explain why we make specific choices. One file tells the AI to check existing tables before creating new ones. It forbids duplicating user data.
- context.md: This file explains our specific terms. It tells the AI how our unique words relate to each other.
- plot.md: This provides a high-level map of the project and how pieces connect.
- Mandatory Tests: Every new API route requires test cases.
This changed everything. Once, the AI changed a shared utility function. The change broke eight other parts of the system. The test suite caught it immediately. The AI saw the failure and fixed its own mistake by creating a version that handled both old and new requirements.
Without tests, that bug would have reached production.
Treat AI like a new developer. You do not blame a new hire for not knowing your codebase. You provide documentation and onboarding. We did the same for the AI.
Our structure:
- docs/context.md: Project terms and connections.
- docs/plot.md: High-level codebase map.
- docs/adr/: Specific rules like table creation and API structure.
Three rules for your team:
- Be specific in ADRs. Use clear instructions instead of vague advice.
- Make docs authoritative. Tell the AI these rules come first.
- Turn mistakes into rules. Every time the AI fails, write a new ADR.
This system does not make AI perfect. It makes AI predictable. We want a codebase where AI is consistent so the team moves faster.