Honest Mistake or Deceptive Marketing? The Ethics of Undisclosed Benchmark Contamination

We have quietly agreed to let a single number stand in for intelligence. A score on a leaderboard now moves purchasing decisions, anchors research citations, and shapes the public story of which lab is winning. So when that number turns out to be partly a reflection of the test itself, we are forced to ask not only what went wrong, but what we were trusting in the first place.

The Eight-Point Drop That Shouldn’t Exist

When Scale AI rebuilt one of machine learning’s most-cited reasoning tests — recreating the grade-school benchmark GSM8k from scratch as a fresh set called GSM1k, same difficulty, none of the same questions — they were not trying to embarrass anyone. They were running a controlled check. A model that genuinely understands arithmetic should not care whether the questions are old or new. Yet on the fresh test, accuracy fell by up to eight percentage points on some model families, with the Phi and Mistral lineages overfitting systematically while frontier models barely moved (GSM1k study). The same work found a faint fingerprint of partial memorization: a modest correlation between how readily a model could reproduce the original items and how far its score dropped.

Eight points does not sound like a scandal. But the number does something quietly unsettling — it tells us that a benchmark we trusted to measure reasoning was, in part, measuring recall. The score was never only about capability. It was also about exposure. And once you accept that a leaderboard can reward Benchmark Contamination — the leakage of test data into training data — the comfortable story that these numbers reflect intelligence begins to fray.

Two Careful People, One Result, Opposite Conclusions

Hand that eight-point drop to two honest, well-informed people and they will reach opposite verdicts.

The first reads it as reassurance. Eight points is modest; the strongest models barely flinched; and most contamination is an accident of how these systems are built. Test sets drift onto the public web and get swept into Common Crawl, which makes up more than eighty percent of the text behind models like GPT-3 and LLaMA (C4 corpus study). Nobody decided to train on the answer key; the leakage is a byproduct of scraping the internet, not a strategy.

The second reads the same number as an alarm. A modest average hides immodest extremes: specific families overfit systematically, the memorization signal is real rather than noise, and contamination is widespread enough that one analysis summarized by DeepLearning.AI estimated nearly a third of a popular benchmark’s questions were compromised, with some models scoring around ten points higher on the contaminated items than on clean equivalents. From this angle the score is not measuring what it claims, and the people selling it have every reason not to look too closely.

Both readings are argued in good faith. And that is exactly where the trouble begins.

What Both Readings Quietly Agree On

Notice what the optimist and the pessimist share, beneath their disagreement. Both treat the number as a verdict on intent. The optimist points to the smallness of the gap as evidence of innocence; the pessimist points to its largeness as evidence of bad faith. Each believes that if we could only measure the contamination precisely enough, the size of the distortion would tell us whether anyone did anything wrong.

This is the assumption almost nobody examines: that culpability can be read off magnitude — that a small gap means honest accident and a large one means deliberate deception. It is a deeply human instinct, the same one that lets us forgive a small lie and condemn a large one. Applied to a benchmark, though, it smuggles in a claim it cannot support.

The Gap Measures Memory, Not Motive

Here is the flaw both readings inherit: the percentage measures memorization, and memorization is a symptom, not a motive. The same eight points can be produced by a test set that drifted innocently into a web crawl, or by a lab that knew the questions and trained on them anyway. The distortion looks identical from the outside. The number records what the model absorbed; it is silent on how, and on whether anyone chose it.

Worse, for the models most people actually rely on, we cannot interrogate the difference. Canary strings — deliberate markers planted in test data — only work if you can read a model’s internal probabilities, which the black-box, API-only models sold as products do not expose. No detection method reliably separates contaminated items from clean ones (Data Contamination Survey). This is the strange asymmetry at the center of the field. In classical machine learning, Data Leakage is a known discipline with real defenses: careful Cross Validation, vigilance against Target Leakage and Temporal Leakage, and open libraries like Deepchecks that flag train-test overlap before it poisons a result. The leakage that inflates the benchmarks deciding reputations and budgets has no comparable leakage audit available to anyone outside the lab.

So when we ask whether benchmark contamination is an honest mistake rather than scientific misconduct, the number we keep pointing to cannot answer. And for the commercial models that matter most, nobody outside the company can answer either.

Where an Accident Becomes a Decision

If the contamination itself does not settle the ethics, what does? The answer, I think, is disclosure — what a lab knew, and what it chose to say.

Consider two documented episodes. In April 2025, Meta submitted an experimental, conversation-optimized variant of Llama 4 Maverick to the public LMArena leaderboard, where it landed near the very top, while the open-weights model the public could actually download ranked far below it; LMArena then changed its rules to require variant disclosure and reproducible prompts (TechCrunch). Meta framed the variant as a legitimate experiment. Separately, Epoch AI built the FrontierMath benchmark with funding from OpenAI, which also had visibility into the problems and their solutions — an arrangement not disclosed until OpenAI’s o3 launch in December 2024, and resting on a verbal agreement not to train on the material (TechCrunch).

In neither case does a contamination percentage carry the moral of the story. What does the work is the gap between what was known and what was shown. That is why the sharpest version of the question is not about leakage at all but about presentation: is undisclosed benchmark contamination a form of deceptive marketing? When a number is placed on a slide to sell a model, and the conditions that produced it are withheld from the people meant to trust it, the omission does the persuading.

Thesis: Undisclosed benchmark contamination crosses from honest mistake into deceptive marketing not at any threshold of percentage points, but at the moment a party who could disclose what the number rests on chooses silence instead.

The contamination is often genuinely accidental; the web is messy, and crawls are blunt instruments. The deception, when it exists, lives one layer up — in the decision to present a compromised or unrepresentative number as if it were clean, and to let the audience supply the trust that the disclosure should have earned.

The Cost That Never Reaches the Leaderboard

No benchmark, contaminated or pristine, can register what matters most here. A score can measure what a model remembers; it cannot measure what is lost when the people who rely on it stop believing it.

Benchmarks work only as a commons — a shared instrument that rivals agree to be measured by, so that comparison means something at all. Every undisclosed contamination spends down that shared trust, and trust, unlike accuracy, does not recover with the next model release. The researcher who built a year of work on an inflated result, the buyer who staked a decision on a figure that quietly included the answer key, the public told that one lab had pulled ahead — none of them appear in the metric. The machine optimizes for the score. We are the ones left to account for what it meant.

Where This Argument Could Break

This case rests on a claim that may not hold forever: that intent cannot be recovered from the outside. If reliable contamination detection for black-box models arrived — a way to prove from the output alone that a specific test had been memorized — then magnitude might genuinely stand in for motive, and my insistence on disclosure would lose some of its force. And there is a more hopeful way the argument dissolves: if leaderboards universally demanded provenance, as LMArena began to after the Maverick episode, the option of strategic silence would simply close. The ethical weight would shift from individual conscience to enforced norm — which is, perhaps, where it belonged all along.

The Question That Remains

We keep treating benchmark contamination as a measurement problem, something cleaner tests will eventually fix — but the number was never the hard part. The hard part is that we built a shared system of trust and then made it optional to tell the truth about what the numbers contain. If a score can be honest only when its makers choose to be, and nothing outside the lab can check them, what exactly are we measuring when we crown one model over another?