Choosing a Model Means Inheriting a Training Run You Can't Patch

The prompt had been stable since March. Same system message, same few-shot examples, same endpoint. Then one Monday the model started refusing a category of request it had handled cleanly for months — and on the requests it still answered, the formatting drifted just enough to break a downstream parser. You open the deploy history out of reflex. Nothing shipped on your side. No commit, no config change, no dependency bump in the lockfile. The thing that moved is upstream, inside weights you have never seen, shipped by a vendor who sent no changelog — because as far as the API contract goes, nothing changed.

The version string is identical. The behavior is not.

How a model behaves is the residue of how it was trained, and training is something you inherit — not a setting you configure. Pre-training fixes what the model can do; fine-tuning and alignment only decide which of those capabilities surface. The assumption that breaks is the one every dependency taught you: that you can read a component, pin it, and patch it to fit. A base model gives you none of the three.

The Build Pipeline That Never Rebuilds

Every model your team calls was built in a strict sequence, and if you have shipped CI/CD you already have most of the mental model. Pre Training compresses trillions of tokens of text into a statistical engine that predicts language. Fine-tuning adapts that engine to a task. Alignment shapes its manners — how it refuses, hedges, and formats. Three stages, each consuming the output of the last. That part maps.

Here is where the analogy stops paying rent. Your build pipeline is reversible. You can check out an old commit, rebuild from source, and get the same binary — deterministic output from deterministic input. A training pipeline has no source to rebuild from. Each stage modifies the weights in place, and there is no earlier checkpoint the vendor hands you. Pre-training set a ceiling on what the model represents well; everything after redistributes behavior underneath that ceiling. When a stage goes wrong, you do not get a compiler error. You get a model that is subtly worse in ways your logs will not name.

Mental Model Map: LLM Training as an Inherited Artifact From: A model is a dependency you read, pin, and patch like any library Shift: Its capabilities were frozen in an irreversible training run before your first call To: You inherit a vendor’s training decisions and can only tune behavior beneath a fixed ceiling Key insight: You are not configuring the model — you are living inside choices someone else already made and cannot undo.

Your evaluation instincts still apply — but their object changed. You still benchmark a candidate before adopting it, still read the docs, still weigh the vendor’s stability. Do all of that. Just know that the docs describe the API surface, not the behavior, and the behavior is the thing that will page you at 2 a.m. In practice, this means your adoption checklist needs a row it never had for a database or a queue: how does this vendor communicate weight changes, and what is your plan when they do not? For the mechanism underneath — how prediction at scale becomes a language engine — Mona’s explainer on how pre-training works has the detail this bridge deliberately skips.

What Fine-Tuning Cannot Reach

The instinct is reasonable and almost always wrong on the first try: the model is close but not right, so you reach for the thing you reach for with any dependency — configure it, patch it, make it fit. In model terms that is Fine Tuning, and it works beautifully for the problems it actually solves and burns weeks on the problems it does not.

Fine-tuning is behavioral calibration, not knowledge injection. It teaches the model which outputs to favor — a format, a tone, an instruction-following pattern — by nudging weights on a few hundred to a few thousand examples. What it does not do is add a capability the base model never learned during pre-training. If the domain was thin in the training corpus, the representation is thin, and no volume of fine-tuning examples conjures knowledge that was never compressed into the weights. Scaling Laws describe why: the base model’s competence ceiling was set by how much of the world’s structure got compressed, how cleanly, at what scale — long before you showed up with your dataset.

That last point has a name and a cost. Push fine-tuning hard enough to force a real behavioral change and the model can lose capabilities that depended on the prior weight configuration — general reasoning, cross-domain coherence, safety behaviors — with nothing in your training loss to warn you. Mona’s analysis of catastrophic forgetting covers the geometry and the mitigations that slow the problem without solving it. Before you spend the GPU budget, sort the gap into one of two bins. If the model can do the thing but does it in the wrong shape, fine-tuning — or often just a better prompt — is the right tool, and the fine-tuning primer plus the LoRA-versus-full-tuning trade-off will tell you how. If the model cannot do the thing because it never learned the domain, fine-tuning is the expensive wrong answer, and the fix lives outside the weights — retrieval, a different base model, or a scope change. In practice, the most costly mistake in this whole area is spending a fine-tuning cycle to close a pre-training gap.

The Version Bump With No Changelog

Here is the shift that catches senior engineers hardest, because it violates a contract they have trusted their entire career. You know how a dependency upgrade works. Semver tells you the blast radius, the changelog tells you what moved, and you pin the version so nothing moves until you say so. A hosted model breaks every clause of that contract while keeping the API signature byte-for-byte identical.

The reason the change hides is that most behavioral shifts you notice come from the alignment layer, and alignment is the least legible stage of the pipeline. RLHF and its successors tune a model’s personality — refusal boundaries, verbosity, how much it hedges — by optimizing against a Reward Model Architecture that compresses human preference into a single score. When a vendor retrains or swaps that layer, the refusal boundary moves, the hedging changes, the format drifts. None of it shows up as an error. None of it is in a changelog, because from the vendor’s side the endpoint still returns a well-formed response. Dan’s account of how alignment methods shifted from PPO to newer optimizers shows how fast this layer is moving — which is exactly why the behavior under your API is less stable than the version string suggests.

Shift Diagram: Upgrading a Dependency vs Inheriting a Model Update Classic: read source → check changelog → pin version → upgrade on your schedule AI: vendor retrains upstream → weights change → behavior shifts with no version signal

You cannot read the source, either. A library you can open and inspect line by line until you know exactly what it does. A model’s behavior is the uninspectable residue of a training run you will never see — no source, no readable diff, no way to selectively invalidate one behavior. What replaces source-reading is observation: golden-set tests you run against the live endpoint on a schedule, so a silent behavioral shift shows up in your CI instead of in a customer complaint. In practice, this means the safety net for a hosted model is not pinning — it is monitoring, because pinning is a promise the vendor never made you.

When a Benchmark Lies About Your Workload

The last inherited assumption is the one that shapes the decision before you write any code: pick the model that scores highest. It feels like sound engineering — you compare candidates on a benchmark and take the winner. For base models, benchmark superiority is a weak signal about your workload, and treating it as a strong one is how teams pick the wrong model with full confidence.

Two things break the “higher score, better fit” reasoning. First, the frontier is flat. As of March 2026, six frontier models sit within 0.8 points of each other on SWE-bench Verified — a spread narrow enough that leaderboard rank is mostly noise for a decision that will hinge on your latency, cost, and domain fit instead. Second, and more fundamental, benchmark scores measure aggregate loss, and aggregate loss is not your task. Only about 39% of downstream tasks scale predictably with the underlying metrics; the rest show irregular, sometimes non-monotonic behavior. A model can top the board and still underperform on your narrow slice, because the board never measured your slice.

The transfer instinct here still holds in form — you should absolutely benchmark before you commit. Just change what you benchmark on. A public leaderboard tells you the vendor did their homework; a hundred examples from your own production distribution tell you whether this model can do your job. Mona’s breakdown of what scaling laws actually predict explains why the curve is real for pre-training loss and a bet for everything downstream, and her survey of the scaling walls shows why raw scale stopped being the thing that separates models. In practice, this means your evaluation harness matters more than the leaderboard you started from — build the harness on your data before you sign anything.

Before You Commit to a Base Model

Run these against your own stack before the base model becomes load-bearing. They are the questions the leaderboard cannot answer for you.

If more than one of these has no answer, you are not choosing a model yet — you are inheriting one and hoping.

Stop treating a base model like a library you can read, pin, and patch; treat it as a vendor’s training run you inherit whole, tune only at the edges, and watch continuously. Before your next model decision, write the golden-set test first and run it against the live endpoint — that harness, not the leaderboard, is what tells you when the ground moved. For the full prerequisite map and the common questions, see the LLM training topic hub.