You built an internal tool that lets a team run 28 unit tests on an LLM-driven feature without ever calling the model’s API. You did it by wrapping the model in a fakeable interface and adding three layers of deterministic, heuristic and LLM-based evaluation.

Standard assertions break the moment an LLM generates prose. The same prompt can yield a different sentence on each run, so assertEqual(output, expected) flags a failure even when the model behaved correctly. Most engineering groups either ship the feature with no verification or try to test the model itself, treating a constantly shifting target as if it were a static library.

Why the problem matters

LLMs now sit inside customer-facing workflows—email outreach, support replies, content generation. A single hallucinated fact or a leaked identifier can damage brand reputation, expose private data, or trigger compliance violations. Without a reliable test strategy, teams waste time chasing flaky failures or ship bugs that only surface in production.

The approach: shrink the model’s responsibility

The first step was to limit what the LLM actually does. In the author’s system the model drafts outreach messages only. All routing logic, state management and safety checks remain in ordinary code. By confining the model to a single, well-defined output, the surrounding system stays deterministic and testable.

To make that possible, the LLM sits behind a provider interface, allowing a fake version to be used in tests. In production the implementation calls the external API; in the test suite a lightweight fake returns a canned response. Because the rest of the code interacts only with the interface, the entire workflow can be exercised by unit tests that never touch the network. The result is a predictable core that the 28 tests verify.

An honest evaluation harness

Even with a narrowed scope, the model’s output remains nondeterministic. The author therefore built a three-layer evaluation harness, each layer handling a different class of risk.

  • Layer 1 – Deterministic checks Simple regular-expression rules catch concrete errors such as a wrong building ID or prohibited tokens. These checks are fast and give a binary pass/fail.

  • Layer 2 – Heuristic checks Scripts look for hallucinated numbers or dates, flagging obvious factual fabrications. They miss false claims that lack numeric cues, and the author openly acknowledges that limitation.

  • Layer 3 – LLM judge A secondary model rates tone and professionalism. Because this step relies on another probabilistic system, it is used only for subjective aspects where deterministic rules would be impossible.

The key to the harness is the dataset used for evaluation. The author encoded known failure patterns—specific traps and domain knowledge—so the harness tests exactly the mistakes that have shown up in practice. It is not a magical “catch-all” but a targeted safety net.

What this means for teams

  • Keep the LLM’s job small. Fewer responsibilities make isolation and testing easier.
  • Put routing, state and safety in code. Traditional logic stays deterministic and fully testable.
  • Expose the model through a fakeable interface. Unit tests run without external calls, keeping the suite fast and reliable.
  • Layer your evaluations. Start with deterministic rules, add heuristics for known hallucinations, and reserve LLM judges for subjective quality checks.
  • State the limits. No layer guarantees perfection; the harness only catches what you explicitly program it to detect.

Counter-point: you still can’t unit-test the model itself

The author concedes that a model is a moving target. Even the LLM judge layer inherits the same nondeterminism it tries to assess. Consequently, the system can never guarantee that every hallucination or policy breach will be caught before release. The approach reduces risk, not eliminates it, and it relies on the team’s ability to keep the evaluation data up to date as new failure modes appear.

Takeaway

You cannot write a classic unit test that asserts an LLM’s exact output, but you can build a system where the model’s influence is bounded, its interface is replaceable, and its output is screened through layered, transparent checks. That combination turns an otherwise flaky component into a predictable part of a larger, testable application.