No Agent Grades Its Own Homework

You ask Claude to review your code. It says the code looks clean. Of course it does. It wrote that code five minutes ago. You asked the author to grade his own paper. He gave himself an A.

AI code reviews work. They fail when you ask the author to review their own work. Quality comes from an architecture where no role checks itself.

Research from 2024 shows a self-preference bias. A model rates its own output higher than others of equal quality. The model recognizes its own style and prefers it.

The loop of "write, then review what you just wrote" is broken. You do not get a review. You get a justification. The agent already decided the code was good. Asking again only confirms that decision.

Follow these rules to build better agent workflows:

  • The reviewer is never the author. Use a different model family for the reviewer to break style recognition.
  • Use a clean context. The reviewer should not see the original implementation prompt or the constraints set by the author.
  • Remove identity. Do not tell the reviewer who wrote the code. The author's identity triggers bias.
  • Avoid over-flagging. AI reviewers often invent problems to look useful. This makes you stop listening to them.

Use the receipt rule to stop false alarms. Every finding must include proof before you see it.

If a reviewer claims a SQL injection risk, they must provide:

  • A grep of the user input.
  • A trace of the query flow.

If the value is a constant, drop the finding. If it comes from an HTTP request, keep it. Proof comes before judgment.

For critical findings, use a panel of skeptics. Their job is not to confirm the bug. Their job is to refute it. They must try to prove why the finding is not a bug. If a majority cannot tear the finding down, only then does it pass.

Truth comes from contradiction, not self-declaration.

Build a system where roles never overlap:

  • The writer writes code.
  • The tester writes tests from the spec only.
  • The reviewer did not write the code.
  • Objective gates like linting and tests must pass before any human or LLM looks at it.

A corrector that corrects itself corrects nothing. The quality of an AI review depends on how many times you stop it from grading itself.

Source: https://dev.to/ohugonnot/no-agent-grades-its-own-homework-8lb

Optional learning community: https://t.me/GyaanSetuAi