Query Key Value

Query Key Value (QKV) refers to the three linear projections applied to input embeddings in transformer attention, enabling each token to search for, match against, and retrieve relevant information from every other token in a sequence.

What It Is

If you’re studying how transformers process language — or trying to understand what happens between raw embeddings and the attention scores everyone talks about — QKV is the mechanism that makes it all work. Without it, a model would treat every token in a sentence identically, with no way to figure out which words matter most to each other.

Think of it like a library search. You walk in with a question (the Query). Every book on the shelf has a label describing its contents (the Key). When your question matches a label, you pull out the actual book and read it (the Value). In transformers, every token simultaneously plays all three roles — asking questions, offering labels, and providing content — which is what makes self-attention so powerful.

Here’s the mechanics. Each input embedding gets multiplied by three separate learned weight matrices: W_q, W_k, and W_v. These produce three vectors per token — the Query vector, Key vector, and Value vector. According to Vaswani et al., attention scores are then computed using the formula Attention(Q,K,V) = softmax(QK^T / sqrt(d_k)) * V. The dot product between Q and K measures how relevant two tokens are to each other. The sqrt(d_k) scaling factor prevents the dot products from growing too large, which would push softmax into regions where gradients nearly vanish. The resulting weights are applied to V, producing a context-aware output for each token.

In multi-head attention, this entire process runs in parallel across multiple “heads,” each with its own set of W_q, W_k, and W_v matrices. According to D2L, each head learns to attend to different types of relationships — one head might track syntactic structure while another captures semantic similarity. The outputs from all heads get concatenated and projected back to the model’s hidden dimension.

This QKV mechanism is the mathematical bridge between embeddings (static representations of tokens) and attention (dynamic, context-dependent representations) — exactly the transition covered in the parent article on the math behind transformers.

How It’s Used in Practice

Most people encounter QKV indirectly every time they use a large language model. When you type a prompt into ChatGPT, Claude, or any transformer-based tool, QKV projections are running at every layer of the model, deciding which parts of your input matter most for generating each output token. The reason a model can understand that “bank” means a financial institution in one sentence and a riverbank in another comes down to how Q, K, and V vectors create different attention patterns depending on context.

If you’re building or fine-tuning models, QKV becomes more direct. Techniques like Multi-Query Attention (MQA) and Grouped Query Attention (GQA) modify the standard QKV pattern to reduce memory usage during inference — they share Key and Value projections across multiple Query heads instead of giving each head its own full set.

Pro Tip: When reading transformer architecture papers, look at the dimension of d_k (the Key vector size) first. It controls the computational cost of attention and tells you how much capacity each head has for distinguishing between tokens. The original transformer used d_k = 64, but modern architectures vary this significantly.

When to Use / When Not

Common Misconception

Myth: Query, Key, and Value are three different types of information stored separately in the model, like three different databases. Reality: All three are derived from the same input embedding through different learned projections. A single token produces a Q, K, and V vector simultaneously. The “separation” exists only because each projection learns to extract a different aspect of the token’s meaning — one for asking, one for being found, one for being read.

One Sentence to Remember

Query asks “what am I looking for?”, Key answers “here’s what I contain,” and Value delivers “here’s my actual content” — together they let every token in a sequence decide how much to pay attention to every other token, which is the core operation that makes transformers work.

FAQ

Q: What is the difference between Query, Key, and Value in attention? A: Query represents what a token is searching for, Key represents what a token offers for matching, and Value holds the content that gets passed forward when a match is strong.

Q: Why does the attention formula divide by the square root of d_k? A: Scaling prevents dot products from becoming extremely large in high dimensions, which would cause softmax to produce near-zero gradients and stall training.

Q: Can QKV be used outside of transformers? A: Yes. The QKV attention pattern appears in graph neural networks, vision models, and retrieval systems, though transformers remain its most common application.

Sources

• Vaswani et al.: Attention Is All You Need - Original 2017 paper introducing the transformer architecture and the QKV attention mechanism
• D2L: Dive into Deep Learning — Queries, Keys, and Values - Interactive textbook explaining QKV with code examples and visualizations