Query Transformation

Query transformation is the pre-retrieval step in a Retrieval-Augmented Generation pipeline where the user’s original question is rewritten, expanded, or abstracted before vector search runs, so the retriever finds the right documents.

What It Is

A RAG system is only as good as the search step in the middle. If a user types a vague question, uses different vocabulary than the source documents, or asks something multi-hop, the vector search returns weak matches and the model answers from thin context. Query transformation is the fix — a small step that sits between the user’s input and the retriever, reshaping the question into one or more forms that are more likely to find the relevant passages.

Four families dominate the technique. Hypothetical Document Embeddings (HyDE) asks the model to draft an imagined answer first, then searches the vector store using that draft instead of the question itself. Multi-Query expansion generates several rewordings of the original question, runs each search, and merges the results. Step-Back Prompting zooms out, asking a more abstract version of the question to retrieve background concepts before drilling into specifics. Decomposition splits a complex multi-hop question into sub-queries that each get their own retrieval pass.

Each family targets a different failure mode. Short queries fail because they don’t carry enough signal — HyDE compensates by inventing the missing context. Ambiguous queries fail because one phrasing only hits one phrasing — Multi-Query covers more of the embedding space. Specific factual questions sometimes fail because the supporting concept lives in a different part of the corpus — Step-Back surfaces that context first. The choice depends less on which technique is “best” and more on which failure mode keeps showing up in production logs.

How It’s Used in Practice

Most teams meet query transformation through a RAG framework rather than building it from scratch. According to LangChain Docs, the MultiQueryRetriever generates several query variants from a single user question and returns the unique union of their search results — making it close to a one-line addition on top of an existing retrieval setup. According to LlamaIndex Docs, the equivalent abstraction is HyDEQueryTransform paired with TransformQueryEngine, which wraps any retriever and applies the transformation transparently. Both patterns show up in production RAG chatbots, internal Q&A tools, and document search assistants where users phrase questions in their own words and the system has to find the right passage anyway. The transformation step is invisible to the user; only the better answers are.

Pro Tip: Log both the original query and the transformed query (or queries) for every request from day one. When retrieval starts returning irrelevant results, you need to know whether the model invented a misleading rewrite or whether the embedding similarity itself was the bottleneck — and you can’t reconstruct that after the fact.

When to Use / When Not

Common Misconception

Myth: Query transformation always improves retrieval recall. Reality: It adds one or more LLM calls before the retriever runs, and on novel or sparsely-covered topics the model can hallucinate misleading expansions — a documented HyDE failure mode where the imagined answer pulls the search away from the truth instead of toward it. Measure recall before and after; don’t assume.

One Sentence to Remember

Query transformation trades a small extra LLM call for a much better chance the retriever surfaces the right document — pick the variant (HyDE, Multi-Query, Step-Back, or Decomposition) that matches the failure mode you actually see in your logs.

FAQ

Q: Does query transformation slow down RAG? A: Yes. Each transformation needs at least one extra LLM call before retrieval. Most teams accept the latency hit because the recall improvement on real user questions outweighs the wait, especially in chat or Q&A use cases.

Q: Which query transformation technique should I start with? A: Multi-Query expansion is the safest default — it generates several phrasings and merges results. HyDE works better for very short queries, and Step-Back helps when the supporting context lives elsewhere in the corpus.

Q: Can I combine multiple query transformation techniques? A: Yes. RAG-Fusion combines multi-query expansion with reciprocal rank fusion, and production systems often route different intents to different transformers. The trade-off is added latency, prompt complexity, and harder debugging when retrieval misbehaves.

Sources

• RAG Survey: Retrieval-Augmented Generation for Large Language Models: A Survey — Foundational survey covering pre-retrieval, retrieval, and post-retrieval enhancements in RAG architectures.
• LlamaIndex Docs: Query Transformations — Reference implementation patterns for HyDE, multi-step, and sub-query transformation in production RAG pipelines.