The Write Succeeded. Retrieval Won't: Where RAG Quality Is Decided

On Monday the legal team dropped six hundred scanned 1990s contracts into your ingestion pipeline. Every write returned a clean 200. The vectors are in the store, the row count matches, the dashboard is green.

By Tuesday the rolling parser bill had tripled, an answer in a customer session quoted a paragraph from a different tenant’s policy, and a multi-hop compliance question — “which counterparties of Subsidiary X are also exposed to Sanctioned Entity Y through any chain of ownership?” — came back with three plausible chunks and no chain. The product team asked, calmly, whether RAG was working.

Here is the part that breaks the old instinct. Nothing failed at query time. The retriever did exactly what it was told. Every one of those problems was written into the store on Monday, hours before anyone typed a question — and no better index, embedding model, or reranker can pull them back out.

Retrieval quality is set before the first query runs. Parsing, chunking, and the metadata schema you commit at ingest decide what is ever findable — the index and the query language only ever work with what those steps preserved. The instinct that betrays you is treating the vector store like a database, where a successful write means the record is retrievable. It doesn’t. A clean write on ingest guarantees nothing about whether a query will surface that content, ever.

Why Your Retrieval Bug Was Written on Monday

The parse that turns a PDF into tokens is the most consequential step in the pipeline, and it runs before anything you would call retrieval.

Your ETL instincts still apply here, and they apply hard. Garbage in, garbage out was always true; RAG just moves the garbage upstream and hides it. A bad extraction is not a transient error a retry will fix — it is a defect baked into every chunk, every embedding, and every answer that touches that document from then on. That old instinct — a clean write means a retrievable row — no longer holds.

The trap is that parser accuracy reads like a single reassuring number. On clean printed text, classical OCR like Tesseract still hits 98-99% accuracy. Point the same engine at scanned PDFs and it drops to 90-95%, at complex layouts to 70-85%, at handwriting to 50-80% (Koncile). A 95% parser sounds fine in aggregate — until you remember it is the upstream layer for chunking, embedding, and retrieval, each of which is also lossy. Stack four lossy stages and “95% accurate” is where a merged-cell table quietly becomes paragraph soup. The model gets blamed for hallucinating when the Document Parsing And Extraction step handed it an incoherent document to begin with.

Mental Model Map: Knowledge Retrieval Systems From: A vector store is a database — a successful write means the content is retrievable. Shift: Retrieval quality is decided at write time; the query only sees what parsing, chunking, and metadata already preserved. To: Ingest is the layer you engineer for correctness; the index and query are downstream consumers of its output. Key insight: You cannot query your way out of a fact your parser destroyed.

The good news is that the write-time layer is now something you can shape rather than accept. Small specialist parsers have overtaken frontier models on real documents — sub-1B vision-language models like MinerU 2.5 and GLM-OCR now top the OmniDocBench leaderboard while the giant general-purpose models sit below them, and self-hosting the specialist runs orders of magnitude cheaper per page. If your ingestion still routes everything through the biggest available model because “the biggest model is safest,” the 2026 document-parsing benchmark shake-up has already moved the cost-quality math against you. The mechanism behind why the bottleneck shifted from characters to 2D structure is in MONA’s parsing-limits explainer; the spec-first way to route documents by complexity instead of paying premium rates on all of them is MAX’s parsing-pipeline guide.

In practice, this means your first RAG bug budget belongs at ingest, not at the retriever. Profile your document mix — what percentage is born-digital text, scanned, table-heavy, slide deck — before you tune a single query parameter.

The Index Is Cheap. The Ingest Is Not.

In classical data systems the index is the expensive, durable asset and the query is disposable. RAG inverts that, and the inversion changes how you plan the whole architecture.

The vector engine is the swappable part. You can migrate from one store to another in an afternoon because the embeddings are portable and the query API is nearly commodity. What is expensive to rebuild is everything the ingest produced: the parsed layout, the chunk boundaries, and — if you went there — the extracted graph. Those are the artifacts that cost real money and real time, and they are the ones a schema change forces you to regenerate.

Knowledge graphs make the asymmetry impossible to ignore. A Knowledge Graphs For RAG pipeline asks an LLM to read every chunk and extract entities and typed relationships, then summarizes communities on top — a multi-pass job whose token cost scales with recursion, not document size. One team scaled the pattern to a real enterprise corpus and paid around $33,000 to index it once (Graph Praxis). That is not a query cost you amortize; it is a write cost you pay again every time the schema or the corpus shifts underneath you. The reason newer variants like LazyGraphRAG defer summary generation to a fraction of the full indexing cost is exactly this: the field is optimizing the write phase, because that is where the bill lives. MONA’s indexing-cost breakdown traces why the cliff is structural rather than a tuning problem.

There is a second cost the graph hides. Auto-extracted graphs invent relationships that were never in the source — the hallucinated-edge rate lands in the low single digits — and a wrong edge is not a retrieval miss, it is a confidently traversable path to a false answer. Before you take on the extra moving parts, MONA’s prerequisites explainer is blunt about the rule: build the vector baseline first, measure where multi-hop queries actually fail, and only then add a graph. The spec-first build order for wiring one is MAX’s GraphRAG pipeline guide.

In practice, this means you budget and version the ingest artifacts as your durable asset and treat the vector database as a deployment choice you can reverse.

A Filter Is Not a WHERE Clause

The first time you point a predicate at a vector index, the API looks exactly like SQL — and that resemblance is the most expensive assumption in the stack.

Your schema-before-load discipline transfers cleanly. You still index the fields you will actually filter on, and you still declare types up front. That instinct is correct and you should keep it. What breaks is everything you assumed about what a filter is. The assumption that the predicate runs like a deterministic query? That assumption is wrong.

Every modern store with Metadata Filtering — Qdrant, Weaviate, Milvus, pgvector, Pinecone — implements one of three strategies behind an identical SDK call: post-filter, pre-filter, or filtered-HNSW. Post-filtering runs the search first and drops the losers, so a selective filter can return fewer results than you asked for. Pre-filtering computes the allow-list first, but if that set is small and disconnected from the dense regions of the graph, recall collapses and the API still hands you ten results as if nothing happened. Filtered-HNSW adds extra edges to keep the graph connected under the predicate. That is three different bets about how a predicate deforms an index — and the store picks one by heuristic, on every call, without telling you. Older pgvector builds silently returned under-filled result sets; Weaviate flips to brute-force scan around a fifteen-percent match rate. The failure is invisible because the result count never changes. MONA’s pre-filter-versus-post-filter explainer is the mechanism, and it is why the benchmark that matters in 2026 is filtered recall under tenant isolation, not raw throughput.

Then there is the assumption that costs the most in production. Adding a new filter field is not ALTER TABLE ADD COLUMN. You cannot backfill a metadata dimension into vectors already written and expect the index to honor it — the payload has to be present when the vector is indexed, which for an existing corpus means a re-index, not a migration. Plan the filter schema before you load anything, the same way you would plan a partition key you can never change.

That distinction is not academic. Cross-tenant leakage is a security incident, and OWASP’s 2025 Top 10 for LLM Applications added a distinct category for exactly this failure class. The full argument for why a filter can never be a permission boundary is in ALAN’s permission-leakage piece, and the spec-first way to encode tenant, time, and permission as an explicit contract is MAX’s metadata-filtering guide.

In practice, this means you test recall at the actual predicates production runs, feed the maximum realistic filter cardinality in staging, and enforce tenancy at the partition, never at the tag.

When the Page Is a Picture, Not Text

The last place the write-time rule bites is the one that looks most solved: turning a document into something searchable when the meaning lives in a chart, not a sentence.

The default playbook — OCR the page, extract strings, chunk, embed — throws away the visual structure before the retriever ever runs. The OCR-first instinct breaks down the moment meaning lives in a picture, not a sentence. A quarterly slide deck goes in, the OCR layer flattens the chart to a caption, and every query about that chart comes back empty, because the thing the query was asking about was discarded at write time. The failure is not in the model’s reading. It is in what the pipeline chose to preserve.

Multimodal RAG closes this by treating each page as an image with structure the model reads directly, rather than a string to be extracted first. Late-interaction retrievers embed the rendered page as a grid of patches and match against it — no OCR step to lose the table in. And modality itself becomes a metadata field: “restrict to figure captions from chapter 3” is a filter, which loops you straight back to the discipline from the previous section. The conceptual difference between this and text-RAG-with-images-bolted-on is in MONA’s multimodal explainer, and the 2026 embedding layer has split into open-source late-interaction and hosted single-vector stacks — which makes this a stack decision, not a model decision. The spec-first build is MAX’s multimodal-RAG guide.

In practice, this means that for visually dense corpora you choose the retrieval architecture before you choose the parser — because the architecture decides whether the parser is even in the pipeline.

Shift Diagram: Knowledge Retrieval Systems Classic: store the data → build the index → optimize the query AI: parse and chunk correctly → commit the metadata and graph schema → the query only surfaces what those preserved

The four surfaces above are not one component with four settings. Once your team builds past the first vector index, retrieval becomes graph traversal, parsing, filtering, and modality routing — each with its own bill, its own failure mode, and its own way of breaking the assumptions you carried in from classical storage.

Before You Trust the Store

Before you scale any of this, the questions worth asking are about your own stack, not the vendor’s benchmark.

The reader who wants the full prerequisite map and the common questions this cluster answers should start at the knowledge-retrieval-systems topic hub.

Stop debugging retrieval at the query and start auditing it at the write: the parse, the chunk boundaries, the metadata schema, and the graph are where your answers were decided. This week, take one failing query, trace it back to the ingest step that produced the chunk it should have found, and fix it there — because that is the only place the fix will hold.