𝗜 𝗙𝗶𝘅𝗲𝗱 𝗟𝗟𝗠 𝗠𝗮𝗿𝗸𝗱𝗼𝘄𝗻 𝗘𝗿𝗿𝗼𝗿𝘀 𝘄𝗶𝘁𝗵 𝗝𝗶𝗻𝗷𝗮𝟮 𝗮𝗻𝗱 𝗔𝗦𝗧 𝗣𝗮𝗿𝘀𝗶𝗻𝗴

𝗜 𝗙𝗶𝘅𝗲𝗱 𝗟𝗟𝗠 𝗠𝗮𝗿𝗸𝗱𝗼𝘄𝗻 𝗘𝗿𝗿𝗼𝗿𝘀 𝘄𝗶𝘁𝗵 𝗝𝗶𝗻𝗷𝗮𝟮 𝗮𝗻𝗱 𝗔𝗦𝗧 𝗣𝗮𝗿𝘀𝗶𝗻𝗴

LLMs generate great content but fail at clean formatting.

In our recent project, LLM technical documents had frequent errors. Code blocks lacked closing markers. Unclosed strings crashed our frontend.

We tried prompt engineering first. We told the model to output correct markdown syntax. The error rate stayed at 15%. This broke our automated publishing pipeline.

The problem is simple. LLMs are probabilistic. Markdown requirements are deterministic. You cannot use a soft constraint like a prompt to solve a hard syntax requirement.

We stopped trusting raw text. We built a validation layer using AST parsing.

Our new process works like this:

• We run an AST syntax check to catch missing quotes or markers.
• If the check fails, we sanitize the text.
• If the check passes, we extract the structured blocks.
• We feed those blocks into a Jinja2 template.

This forces the output to follow a fixed structure. The template engine controls the layout, not the LLM.

We also added safety measures to prevent system crashes:

• Exponential backoff: If parsing fails, the system waits before retrying.
• Hard fallback: If all retries fail, we strip the formatting and serve plain text. This ensures the user gets the content even without the style.
• Timeouts: We use a 5-second limit to prevent the pipeline from stalling.
• Error sampling: We log 10% of failures to debug without wasting storage.

The results:

• Error rates dropped from 15% to 0.1%.
• Engineering determinism replaced unreliable prompt engineering.
• The pipeline remains stable during LLM failures.

Stop asking the LLM to format. Build a pipeline to enforce it.

Source: https://dev.to/quarktimes/i-fixed-llm-markdown-errors-with-jinja2-and-ast-parsing-25e0