Decoder-Only Architecture

Almost every large language model shipping in 2026 — GPT, LLaMA, DeepSeek, the text side of Gemini — runs on the same layout choice: one stack, generating one token at a time, with nothing held back in a separate encoder. That choice, not raw parameter count, is why picking or building a foundation model starts with this pattern before any of the other core layouts in the transformer and attention internals theme make sense. It also explains why an entire industry is now debating whether betting everything on one architecture was ever actually a decision.

How to read the decoder-only pattern, mechanism to market

Begin with how autoregressive LLMs generate text token by token — it owns the mechanism, causal masking, next-token prediction, the KV cache, that every later article assumes you already have. Once that lands, why decoder-only beat encoder-decoder answers the question the mechanism alone can’t: why the simpler half of the original transformer, not the full assembly, went on to scale.

When you’re ready to specify a build rather than just read about one, the decoder-only build guide turns the causal mask and the fine-tune-or-not decision into a spec an AI coding assistant can’t misread. For where that spec collides with the current market, DeepSeek MLA, LLaMA 4 MoE, and Nemotron hybrids tracks how the once-uniform decoder-only camp split into dense, MoE, and hybrid variants competing on cost per token. Close with the decoder-only monoculture for the uncomfortable question behind all of it — whether this pattern won on merit or just arrived first.

How decoder-only differs from its closest neighbours

The confusion that costs the most debugging time is treating a decoder-only stack as simply “the decoder half” of encoder-decoder architecture. It is not that simple: an encoder-decoder’s decoder layer still runs cross-attention back to the encoder’s output on every step, while a decoder-only block only ever attends to its own previous tokens. Strip cross-attention out of an encoder-decoder decoder and the result is a broken model, not a decoder-only one — decoder-only is trained and served as its own design, not encoder-decoder with a part removed.

A second confusion sits one layer deeper: “decoder-only” names the generation pattern, not what fills each block. DeepSeek’s MLA variant, LLaMA 4’s MoE layers, and Nemotron’s hybrids, which borrow recurrence from state space models, are all still decoder-only — each generates one token at a time from a single stack, even where mixture of experts swaps the shared feedforward layer for a routed pool of specialists. Judging an architecture’s decoder-only status by parameter count or routing style is the category error the market is currently sorting out.

Common questions about decoder-only models

Q: Should I reach for a decoder-only model when the task needs to see the whole input before producing any output, such as translation? A: Not by default. Decoder-only models generate causally, each token conditioned only on what came before it, which is a poor fit once input and output are genuinely different sequences. Why decoder-only beat encoder-decoder lays out exactly where that trade-off still favors the older layout.

Q: Why do some models marketed as decoder-only still route tokens through mixture-of-experts layers? A: Because decoder-only describes how a model generates text, not how its feedforward layers are built. DeepSeek MLA, LLaMA 4 MoE, and Nemotron hybrids shows dense, MoE, and hybrid variants all keeping the same autoregressive, single-stack generation underneath.

Q: Why does a decoder-only build that looks architecturally correct still fail during training? A: Usually the causal mask, not the overall design. One wrong index lets a token attend to positions it should never see, and the loss curve looks plausible while the model learns the wrong task. The build guide treats the mask as its highest-risk specification for exactly this reason.

Q: Was decoder-only chosen because it was technically the best option, or because it happened to scale first? A: Closer to the second: no rigorous head-to-head comparison picked this pattern, the first version that scaled simply became the default everyone kept building on. The decoder-only monoculture examines what that unexamined bet costs now.

Q: Do I need to understand causal masking before the build guide’s model-selection framework will make sense? A: Yes — the guide assumes you already know why a token can only see what came before it, and spends its own budget on the mask’s implementation and the fine-tune-or-not decision instead. Start with the mechanism first.

Part of the transformer and attention internals theme · closest neighbour: encoder-decoder architecture. New to transformer internals from a software background? Start with the story: Transformer Internals for Developers: What Maps, What Breaks.