Benchmark Datasets

Every claim that a model has gotten better ultimately rests on a benchmark dataset saying so, which is why this topic sits inside the synthetic data theme rather than beside it: once training corpora start absorbing generated text at scale, the yardstick and the thing it measures can quietly start to overlap. A model card citing a state-of-the-art number is making a claim that only holds up if you know which benchmark produced it, how it was run, and whether it has already been gamed. For a developer choosing between models, or reporting a result upward, the score is only as trustworthy as the process behind it.

Reading this topic: from raw score to a defensible claim

Start with what benchmark datasets are and how GLUE, MMLU, and SWE-bench measure LLM performance — it maps how a fixed set of questions with known-correct answers turns into the one number every launch post cites. Read the prerequisites for reading AI benchmark scores alongside it: pass@k and contamination decide whether that number means what the headline implies. Then the technical limits of AI benchmarks closes the diagnostic set — saturation and construct validity are failure modes that survive even a clean contamination check.

When you need to produce a score yourself rather than just read one, the lm-evaluation-harness guide routes MMLU-Pro and GPQA through one harness and SWE-bench through its own Docker setup, and pins the version details a reproducible score depends on. For where the goalposts stand right now, SWE-bench Pro, ARC-AGI-2, and Humanity’s Last Exam covers the suites labs adopted once the older ones saturated. Close with the skeptical read: Teaching to the Test argues that chasing the score itself is a different project from improving the model.

How benchmark datasets differ from the rest of the synthetic data stack

• Benchmark datasets vs synthetic data generation. Generation produces the material a model trains on; benchmarks measure what the trained model can do afterward. The two collide only when a training-scale corpus accidentally swallows benchmark questions — a distinct failure from a badly generated dataset, and one no amount of fixing the generator alone will resolve.
• Benchmark datasets vs synthetic data ethics. A model can top every benchmark it is measured on while trained on data whose privacy and consent story was never checked — a score answers “does it perform,” never “was the data legitimate to use.” Neither question substitutes for the other, and both need separate evidence.
• Benchmark dataset vs evaluation harness. The dataset is the fixed set of questions and answers; the harness is the software that runs a model against them and computes the score. Swapping harnesses without pinning fewshot count, prompt format, and dataset version is a common, quiet source of scores that will not reproduce, and part of why construct-validity problems persist even after a contamination check passes.

Common questions about benchmark datasets

Q: Which article should I read if I only need to cite a benchmark score, not run the evaluation myself? A: The prerequisites for reading AI benchmark scores — it covers what the metric counts, how many attempts the model got, and whether the training data was contaminated, which is exactly what a citation needs to hold up without you running anything.

Q: Do I need to run my own benchmark suite, or can I cite a published leaderboard score? A: Citing is fine if you can verify how the number was produced; if you cannot, the lm-evaluation-harness guide shows what pinning a fewshot count, prompt format, and dataset version actually buys: a score someone else can reproduce.

Q: Why did SWE-bench Verified get retired if it was the standard coding benchmark? A: Models had started scoring well on it for reasons that stopped tracking real coding ability, the same saturation pattern that keeps retiring benchmarks industry-wide. SWE-bench Pro, ARC-AGI-2, and Humanity’s Last Exam covers what labs adopted in its place and why.

Q: How can you tell if a model was trained to game a specific benchmark rather than genuinely improve? A: There is no single tell — it shows up as a gap between benchmark rank and real-task performance, plus training data that overlaps suspiciously with the test set. Teaching to the Test traces how that gap forms and why it is easy to miss on a leaderboard alone.

Part of the synthetic data theme · closest neighbour: synthetic data generation.