Fine-Tuning

Fine-tuning is where developers first get a hand on the LLM training ladder: pre-training happens at a scale almost nobody reading this will ever run, but adapting an existing checkpoint to one task is a decision within reach of a single GPU and a well-curated dataset. That reach is also the trap — the same small dataset that teaches a model your task can just as easily overwrite what it already knew, and a platform market now racing toward sub-$0.50-per-million-token pricing rarely advertises that trade-off. Getting the method, the data, and the failure modes right, in that order, is what separates a specialist model from a quietly broken one.

Reading fine-tuning in build order, not publish order

Start with how gradient updates adapt a pre-trained model to a specific task — it is the only article here that owns the mechanism, and every later piece assumes you already have it. Next, the LoRA vs. QLoRA vs. full fine-tuning comparison turns that mechanism into a method decision before you touch a training script. Read catastrophic forgetting, overfitting, and the hard technical limits of fine-tuning before you ship, not after — it names the failure mode that catches nearly everyone on a first attempt.

Once the method is settled, the Hugging Face PEFT, Unsloth, and Axolotl guide turns the decision into a working pipeline. For the market context behind tooling choices, Together AI’s platform race tracks how fine-tuning got this cheap and what that means for lock-in. Close with the accountability gaps in fine-tuned LLMs — if your fine-tune touches data you did not collect yourself, read it before you deploy, not after.

Fine-tuning versus scaling up, versus the reward model

Two confusions come up more with fine-tuning than with any other stage of training.

The first is reaching for a bigger base model when a fine-tune would do. Scaling laws govern whether more parameters or more training data improve a pre-training run — they say nothing about whether your narrow, well-defined task needs a bigger foundation at all. Most cases that look like “the model isn’t good enough” are format and domain problems a fine-tune fixes in hours; a genuine capability gap is the rarer case that actually calls for a newer base model.

The second is assuming fine-tuning needs a scoring model. It does not. Supervised fine-tuning updates weights directly from labeled examples you already hold; a reward model only enters the picture once you are optimizing against human preferences rather than a fixed dataset — a separate, later stage. Conflating the two leads teams to build reward-model infrastructure for a job a plain fine-tune already does.

A third, more mundane confusion: the fine-tuning method and the fine-tuning platform are not the same purchase. LoRA, QLoRA, and full fine-tuning are choices about which weights move; the services that run those jobs are a choice about whose GPUs do the work. Platform prices have converged near a floor, so the method decision still has to happen first, not the vendor decision.

Fine-tuning questions that come up once you’re building

Q: How much training data do you actually need for a good fine-tune? A: Far less than most teams assume — a small, consistently formatted dataset routinely beats a much larger, noisy one; one comparison found 500 clean examples outperforming a 50,000-row dump with inconsistent structure. The open-source fine-tuning guide covers how to curate and format that set.

Q: Is a LoRA adapter a separate model from the one it was trained on? A: No — a LoRA adapter is a small set of additional weights layered on top of the frozen base model, not a standalone model. Losing or corrupting the base checkpoint breaks every adapter trained against it, a dependency the methods comparison weighs against when full fine-tuning is worth its extra cost.

Q: Does the cheapest fine-tuning platform also produce the best model? A: Not necessarily — price competition has pushed sub-16B fine-tuning below $0.50 per million tokens, but the platform race shows the real differentiation has moved to training speed and inference tooling, not the headline price.

Q: Can fine-tuning remove a bias that came from the base model’s pre-training data? A: Rarely, and often not fully — a fine-tune adjusts behavior on your task’s examples, but biases baked into the much larger pre-training corpus tend to persist underneath it, which is why the accountability-gaps analysis treats the base model’s data as a standing liability, not a solved problem.

Part of LLM training and pre-training · closest neighbour: scaling laws. New to this from a software background? Start with the story: LLM Training for Developers: Which Instincts Help, Which Mislead.