Developers can now keep the price of fixing malformed JSON from large language models (LLMs) under control by capping repair attempts at two, a simple Python loop shows. The pattern—generate, parse, validate, retry, then accept or fail—turns “maybe-valid JSON” into a measurable success rate and prevents runaway token usage.

Why a repair loop matters

LLMs are often instructed to “return JSON,” yet the output frequently contains syntax errors, missing keys, or values that break business rules. A downstream system that expects a strict payload will crash or produce wrong results if fed such output. Without a systematic way to catch and correct these problems, teams either accept a high failure rate or waste money prompting the model repeatedly until it gets it right.

The three validation layers

A reliable loop separates validation into:

  • Syntax – does the string parse as JSON at all?
  • Shape – does the top-level object contain exactly the expected keys?
  • Semantic – do the values respect domain constraints (for example, a “category” must belong to a predefined set)?

By stopping at the first failing layer, the loop can give the model a precise error message, which dramatically improves the chances of a correct fix on the next try.

Minimal Python example

The following code uses only the standard library. It defines a tiny schema (a dictionary with keys category and summary) and a fixed list of allowed categories.

import json

CATEGORIES = {"bug", "feature", "question"}

def validate(raw: str):
    try:
        value = json.loads(raw)
    except json.JSONDecodeError as exc:
        return None, f"syntax: {exc.msg}"

    if not isinstance(value, dict):
        return None, "shape: root must be an object"
    if set(value) != {"category", "summary"}:
        return None, "shape: expected category and summary"
    if value["category"] not in CATEGORIES:
        return None, "semantic: invalid category"
    if not isinstance(value["summary"], str) or not value["summary"].strip():
        return None, "semantic: summary must be text"

    return value, None

The repair loop itself imposes a hard ceiling on attempts:

MAX_ATTEMPTS = 2
raw = model_response   # initial LLM output

for attempt in range(MAX_ATTEMPTS + 1):
    value, error = validate(raw)
    if error is None:
        break
    if attempt == MAX_ATTEMPTS:
        raise RuntimeError(f"failed after {MAX_ATTEMPTS} retries: {error}")

    # Send the error and schema back to the model for a fix
    raw = ask_model_to_repair(raw, error)

If the JSON passes validation on the first pass, the loop exits immediately. Otherwise it retries, feeding the model a concise payload: the original output, the exact error string, and the expected schema. After two failed attempts the process aborts with a clear exception.

Two rules for cost-effective repairs

  1. Keep repair prompts tight – Include only the raw JSON, the error message, and the schema. Adding a long chat history inflates token usage and can confuse the model, pushing the cost per retry higher without improving accuracy.

  2. Separate validation from fact-checking – The parser can detect structural problems, but it cannot verify that a “summary” statement is true. Fact-checking belongs in a later stage; attempting to “repair” a false claim merely makes the lie look cleaner.

Measuring success

Logging each attempt’s error type (syntax, shape, semantic) and whether the loop succeeded gives a concrete metric: the proportion of LLM responses that become usable within the bounded retries. Teams can track this over time, compare prompt styles, or experiment with different schema definitions.

Potential downsides

Bounding the loop means some malformed outputs will be discarded after the limit is reached, which may increase the overall failure rate if the model’s raw quality is low. In environments where every response is critical, developers might prefer a higher retry ceiling at the expense of extra tokens. The trade-off is explicit: more cost for higher coverage, or tighter budgets with a predictable ceiling.

What to watch next

  • Prompt engineering for repair – Refining the error message format can reduce the number of retries needed.
  • Alternative parsers – Some libraries provide tolerant parsing that can auto-correct minor issues; comparing their cost versus a bounded loop is worthwhile.
  • Model updates – Newer LLM versions may produce cleaner JSON out of the box, potentially allowing the retry limit to be lowered further.

A bounded JSON repair loop gives developers a practical tool to turn fuzzy LLM output into reliable data without spiralling costs. By treating validation as a first-class step and capping retries, teams can keep both budgets and downstream systems happy.