The AI agent I shipped passed 23 unit tests, but within an hour of going live it invented a product feature and quoted a price three times lower than reality. When the user called it out, the bot doubled down, the conversation ended, and I lost a client. The mistake proved that a suite of deterministic unit tests cannot guarantee an agent’s reliability.

Why unit tests fall short for AI agents

Unit tests work for traditional code because the same input always yields the same output. “2 + 2 = 4” is a guarantee you can verify with a simple equality check. An LLM-driven agent, however, shifts its output with the prompt, surrounding context, and the state of any external tools it calls. A test that checks for exact string equality misses hallucinations, tone shifts, or guardrail violations. The silent failure that cost a client shows that you must evaluate the whole interaction, not just isolated functions.

Building an evaluation harness before any feature code

I flipped the development order: design a four-layer evaluation harness first, then write the agent. The harness runs 131 tests in a single pass, costs roughly three cents per run, and finishes in about eleven minutes. I assign each test to the smallest model that can handle it, reserving larger, pricier models for moments they truly add value.

Layer 1 – Tool functionality

The first line of defense checks whether the agent can call its tools correctly. Tests cover successful searches, deliberately malformed queries, and simulated API failures. Because tool usage is largely deterministic—either the request is formed correctly or the API returns an error—simple Python assertions suffice. Catching a malformed request here prevents downstream confusion.

Layer 2 – Instruction following

Next the harness verifies that the agent respects guardrails. A smaller LLM acts as an evaluator, scanning the agent’s response for compliance: staying in character, avoiding prohibited topics, and emitting the required JSON schema. This layer catches semantic drift that unit tests miss, such as slipping into an unintended persona or leaking internal prompts.

Layer 3 – Goal-oriented behavior

The third layer is the most critical. It asks whether the agent actually accomplishes its purpose. For a lead-generation bot, that means confirming it asks the right qualifying questions and escalates to a human when appropriate. I use a reasoning-oriented model here because it can assess the overall flow without inflating costs. If the bot fails to achieve its goal—even while passing the first two layers—it gets flagged for redesign.

Layer 4 – Performance

Finally, the harness records latency and token-generation speed. Slow responses erode user experience, especially in real-time chat. By tracking these metrics alongside functional correctness, I ensure the agent is both accurate and responsive.

Cost-saving choices

The $0.03 per run figure is not a marketing gimmick; it comes from matching test complexity to model size. Deterministic tool checks run on the cheapest runtime, instruction compliance uses a lightweight model, and only the goal-oriented assessments invoke a more capable, albeit pricier, model. This tiered approach keeps the total expense low enough to run the full suite on every code change.

The trade-off: speed versus safety

Introducing a harness added upfront friction. Development cycles stretched, and the launch timeline slipped.

What to watch next

  • Model-driven evaluators: As LLMs improve, the evaluator in Layer 2 can become more nuanced, reducing false positives while still spotting subtle policy breaches.

Takeaway

If you build AI agents for production, a layered evaluation harness is not optional; it is foundational. By front-loading the cost of comprehensive testing—$0.03 per run, eleven minutes per suite—you protect against the silent failures that unit tests simply cannot detect.