Query Transformation

A retriever can only find documents that share vocabulary with the question asked of it, and users rarely word things the way source documents do. That mismatch — not a weak embedding model or a badly tuned index — is what query transformation exists to close: an LLM reshapes the question before it reaches the retriever, without touching a single stored vector or rebuilding an index. That is why it sits at the front edge of the retrieval-augmented generation stack, the lever pulled before search runs, not the one pulled after results come back thin.

Reading query transformation in order: mechanism, build, judgment

Start with what query transformation is and how HyDE, multi-query, and step-back prompting improve RAG recall — it separates the three core rewrite strategies before any guide asks you to pick one. When users ask compound questions a single rewrite cannot cover, query decomposition and routing explains how splitting a question into sub-queries and fusing the results with RAG-Fusion recovers recall a single rewrite loses.

Before wiring anything into production, read the hard technical limits of query transformation — every rewrite pays a latency tax, and one failure mode matters more than the rest: a hypothetical document plausible enough to ground retrieval in fiction. With that ceiling in view, the LangChain build guide wires HyDE and multi-query into a working pipeline, and the HyDE vs. multi-query vs. step-back decision guide turns query shape into a routing rule instead of a guess.

How query transformation differs from reranking and contextual retrieval

Two neighbours get folded into the same mental bucket, and each mixing sends debugging in the wrong direction.

Query transformation acts before retrieval, widening what gets found by rephrasing the question; reranking acts after retrieval, reordering what already came back. A transformation problem shows up as documents that were never retrieved at all; a reranking problem shows up as the right document buried at position eight. Query transformation also targets a different side of the pipeline than contextual retrieval: transformation rewrites the question, while contextual retrieval enriches the document side by prepending context to each chunk before it is ever indexed. One fixes what you ask; the other fixes what got stored.

That two-sided view also explains where production practice moved in 2026 — teams shifted from a fixed rewrite step to router-selected transformation that picks a technique per query instead of running the same one on everything.

Common questions about query transformation

Q: Should I run HyDE on every query by default? A: No — treat it as a routed choice, not a default. Production teams that ran HyDE always-on hit a recall trap on structured corpora and a latency tax on smaller models; selective routing is now the standard practice, reserving the rewrite for the query shapes it actually helps.

Q: Can a rewritten query change what the user is actually asking? A: Yes, and that is the deeper risk, not just latency. An LLM that rewrites a question before retrieval effectively answers a different question than the one typed, often without telling the user it happened — an accountability question worth weighing before shipping the rewrite silently.

Q: Is a failed query rewrite easy to catch before it reaches the retriever? A: Not automatically — a hallucinated HyDE document reads like plausible prose, so nothing flags it before it grounds retrieval in fabricated text. The three structural costs — latency, drift, and hallucination — stay silent until you measure recall directly, not the rewrite’s fluency.

Q: If I add query transformation, does the reranker score the original question or the rewritten one? A: The reranker should always score the original user query, not the rewrite — HyDE retrieves with the hypothetical document, but ranking still has to answer what the user actually asked. Reranking against the rewrite is a common pipeline mistake that quietly drifts relevance.

Part of the retrieval-augmented generation theme · closest neighbour: reranking. Coming to this from a software background? Start with the story: Debugging RAG Failures: Why Developers Need a New Diagnostic Model.