Most RAG-Tutorials enden beim Notebook. Sie laden ein paar polierte PDFs, teilen den Text alle tausend Zeichen auf, stopfen die Fragmente in eine Vektordatenbank und nennen es eine Architektur. An einem Freitagnachmittag läuft diese Demo perfekt. In der Produktion wird dieselbe Pipeline schleichend zu einer Belastung.

Der wahre Flaschenhals in einem Retrieval-System ist selten das Modell oder der Prompt. Es ist die Ingestion. Eine RAG-Pipeline kann nur das abrufen, was ihr zugeführt wurde, und wenn die Zufuhr verrauscht, veraltet oder unvollständig ist, liefert das Modell selbstbewussten Unsinn. Wenn Nutzer sich beschweren, dass der Bot halluziniert hat, liegt der Fehler oft meilenweit stromaufwärts in einer Datenpipeline, die niemand genau überwacht.

Die Whiteboard-Falle

Architekturdiagramme lassen die Ingestion wie einen einzelnen Pfeil mit der Beschriftung „Dokumente → Vektordatenbank“ aussehen. Die Realität ist chaotischer. Quellsysteme ändern sich ohne Vorankündigung. HTML-Layouts werden neu gestaltet. URLs leiten auf generische Landingpages um. JavaScript-Frameworks tauschen den Inhalt nach der initialen HTTP-Antwort aus. Die Ingestion als einmalige Einrichtung zu betrachten, ist der erste Fehler. Es ist ein fortlaufendes Data-Engineering-Problem, das die gleiche Sorgfalt verdient wie jede ETL-Pipeline.

Warum RAG-Fehler meistens Zufuhr-Fehler sind

Stellen Sie sich Folgendes vor: Ein Nutzer fragt Ihren internen Assistenten nach der aktuellen

A static snapshot of an internal wiki is simple mode. Continuously ingesting from the live web is hard. You need to know when a page was last gathered, whether it has changed since then, and how long the information remains valid. Stale data does not always mean a visibly old date. Sometimes a page updates its text but keeps the same URL, so your system never notices without content hashing. Build clear refresh rules based on source volatility. A financial data feed might need hourly checks. A company about page might need quarterly checks. Record timestamps and set time-to-live boundaries, especially if your domain involves regulated or safety-critical guidance where old facts can cause real harm.

5. Duplicate Pollution

Websites are full of repetition. The same product description appears on the category page, the product page, and a promotional landing page. The same press release lives under /news/, /press/, and /blog/. Vector search does not deduplicate automatically. If ten nearly identical chunks sit in your database, they can crowd out diverse, relevant results in your top-k retrieval. You need canonical tracking or content deduplication before embedding. If two chunks say the same thing, keep the authoritative source and drop the copies. Your retriever has limited slots. Do not let them go to waste.

6. Missing Metadata

A vector database without metadata is just a dense text search engine with no memory of context. Smart retrieval depends on filtering and ranking signals that raw embeddings cannot provide. Store the source URL, the capture date, the document category, and the version number. If you ingest API documentation, versioning is essential. Without it, a query might blend v1 and v2 specs into the same answer. If you ingest HR policies, tagging by region or department lets you filter results before they ever reach the model. Metadata turns a text dump into a curated knowledge system.

7. JavaScript Gaps

Modern sites do not ship their content in the first HTML payload. They send a skeleton and hydrate it with JavaScript calls. A basic HTTP request might see nothing but a loading spinner and a layout shell. If your pipeline cannot execute JavaScript, you will ingest blank pages or partial fragments and never realize anything is wrong. Using a headless browser solves the rendering problem but introduces new ones: heavier memory use, slower throughput, and bot detection walls. Choose your trade-offs deliberately, but do not pretend a simple curl equivalent is enough for every source.

A Practical Ingestion Checklist

If you are building or reviewing a RAG feed, start here:

  • Validate source coverage and pagination. A sitemap might list only the first ten articles in a category. Crawl deep and verify that paginated or dynamically loaded content is actually captured.
  • Remove boilerplate before chunking. Strip navigation, ads, footers, and repeated legal disclaimers. If a phrase appears on every page, it is noise.
  • Use structure-aware chunking. Respect headings, bullet lists, and tables. Split on semantic boundaries, not character counts.
  • Attach rich metadata. Include URL, capture date, content category, and version. Make these fields filterable in your retrieval queries.
  • Set refresh frequencies based on data volatility. High-change sources need frequent re-crawls. Static archives do not.
  • Monitor the corpus, not just the job status. A pipeline can exit with code zero while producing garbage. Audit samples of stored chunks regularly for drift and quality.
  • Define rules for versioning and deletions. When a source page is removed, delete its chunks. When it updates, overwrite or version them. Orphaned data is a silent killer.

The Hard Truth About Embeddings

No embedding model, no matter how advanced, can repair a missing document. It cannot guess that a page was updated last week if your feed still holds last year's copy. It cannot infer the context of a table row that got separated from its header by a bad chunk boundary. Embeddings compress meaning, but they do not create meaning where the ingestion layer failed to preserve it.

Retrieval quality starts at the ingestion layer. That layer decides whether your RAG system is a useful tool or just a confident liar with a vector database behind it.

The Real Takeaway

Messen Sie die Qualität der Ingestion nicht allein anhand von Pipeline-Dashboards. Erfolgreiche Jobs und saubere Logs garantieren noch keinen sauberen Korpus. Öffnen Sie die Datenbank und lesen Sie die tatsächlichen Chunks, die Ihre Nutzer abrufen werden. Wenn der Text voller Urheberrechtshinweise, zerrissener Tabellen und veralteter Richtlinienseiten ist, liegt das Problem nicht beim LLM. Korrigieren Sie zuerst den Feed. Alles andere ist nur das Tuning von Datenmüll.