Opening bank statements is nobody's idea of fun. They arrive as scanned PDFs, CSV exports, or XML files dressed up with

Opening bank statements is nobody's idea of fun. They arrive as scanned PDFs, CSV exports, or XML files dressed up with arcane acronyms like OFX. For accountants, bookkeepers, and fintech builders, turning these documents into clean, structured data is a constant headache. When large language models burst onto the scene, they seemed to offer an escape hatch. Just feed the machine a PDF and ask for JSON. What could go wrong?

I learned exactly what could go wrong while building StatementDecoder, a tool designed to convert bank statements into usable data. Like many developers, I assumed the hard part would be teaching the system to read diverse document layouts. I was wrong. Reading the documents was almost trivial. The real nightmare was recognizing when the machine had quietly invented a number or swapped two digits in a transaction amount.

The Demo That Worked Too Well

My first attempt was seductively simple. I piped bank statements directly into an LLM and requested structured JSON in return. The results felt like magic. The model handled different layouts with ease. It read scanned PDFs that standard parsers choked on. It seemed to understand tables, headers, and multi-page statements without explicit instructions. For a few glorious hours, I thought the problem was solved.

Then I tested it against real customer data, and the magic evaporated. UK banks each use their own statement designs, and the differences are not merely cosmetic. Wise statements carry their own formatting quirks. Revolut CSV exports look straightforward until you notice how they handle multi-currency transactions and metadata fields. Old OFX files, a format that genuinely looks like it belongs in the 1990s, throw archaic tag structures and encoding issues at any parser that expects modern markup.

The model still extracted data far better than any off-the-shelf template system. But far better is not good enough when money is involved.

When 99% Accuracy Is a Failure

Here is the fundamental problem with using AI for financial data extraction. If a model processes two hundred transaction rows and gets one hundred ninety-nine correct, the output looks pristine. The JSON is well-formed. The keys and values align. A casual review might show nothing suspicious. Yet if that single error transposes two digits in an amount, flips a deposit into a withdrawal, or shifts a decimal point, your bookkeeping is corrupted. You will not catch it by eyeballing a wall of structured data.

A human reviewing raw JSON rarely spots a swapped digit in a transaction amount. The formatting is perfect, which paradoxically makes the mistake more dangerous. You cannot ship a financial tool that is right most of the time. It must be right, or it must loudly announce that it is unsure.

My initial reaction was predictable. I engineered better prompts. I upgraded to more capable models. I experimented with chain-of-thought reasoning to make the model show its work. None of this fixed the core issue. I was asking the same probabilistic system to generate an answer and then asking that identical system to certify that the answer was correct. That is not verification. That is self-consistency theater.

Let the Math Decide

Bank statements have a feature that most documents do not: built-in arithmetic constraints. The opening balance plus the sum of all transactions must equal the closing balance. Running balances, when present, must line up row by row. These are not stylistic preferences. They are hard rules.

I rebuilt the architecture around this insight. Now, every extraction, regardless of origin, flows through a validation layer before any user sees it. It does not matter whether the data came from an LLM interpreting a fuzzy PDF, an OCR engine reading a scanned page, or a direct CSV parse. The validator treats all sources as equally suspect.

The check is brutally simple. Add every transaction to the opening balance. Compare the result to the stated closing balance. If the numbers do not match, something is wrong. Flag the statement for review. Reject the extraction. Do not let it reach the user.

This single change altered the entire character of the product. The language model no longer needed to be perfect. It only needed to be good enough to produce output that could survive a mathematical trial. The pressure shifted from achieving impossible accuracy in an unconstrained domain to building a tight feedback loop between generation and verification.

Der Validator deckte auch Muster in den Fehlern auf. Bestimmte Dokumenttypen scheiterten konsistent an der mathematischen Prüfung, was mir genau zeigte, wo ich meine Bemühungen investieren musste. Anstatt blindlings das Prompt Engineering flächendeckend zu verbessern, konnte ich erkennen, dass spezifische Bank-Layouts systematische Fehler verursachten.

Code dort, wo Code hingehört, KI dort, wo sie glänzt

Vielleicht war die demütigendste Lektion die Erkenntnis, wie viel von der Pipeline überhaupt keine KI benötigte. Als ich auf unordentliche australische OFX-Dateien stieß, war mein Instinkt, das Problem einfach mit Tokens zu lösen. Ich zog kurz in Erwägung, das fehlerhafte XML in das Modell einzuspeisen und es zu bitten, die Struktur vor dem Parsen zu reparieren. Stattdessen schrieb ich zwanzig Zeilen deterministischen Codes. Er behob die Encoding-Eigenheiten und fehlerhaften Tags sofort, mit null Kosten pro Datei und perfekter Reproduzierbarkeit.

Diese Erfahrung verdeutlichte, wie Extraktions-Pipelines organisiert sein sollten. Es gibt drei unterschiedliche Aufgaben, die nicht vermischt werden sollten.

• Das Modell versteht unordentliche Dokumente. Gescannte PDFs mit verzerrten Tabellen, gemischten Schriftarten und handschriftlichen