Label Flipping
Also known as: label-flip attack, adversarial label noise, training label corruption
- Label Flipping
- Label flipping is a data poisoning attack in which an adversary changes the class labels of selected training samples while leaving the underlying data unchanged, causing a model to learn incorrect input-output associations without any detectable anomaly in the data features.
Label flipping is a data poisoning attack where an adversary changes the class labels of training samples without modifying the data itself, causing an AI model to learn wrong classifications.
What It Is
When a machine learning model learns from labeled data, it trusts that the labels are correct. Label flipping exploits that trust. An adversary gains write access to some portion of the training dataset and changes the class labels on selected samples — marking spam emails as legitimate, flagging safe URLs as malicious, or labeling a stop sign as a speed limit sign — while leaving the actual data content completely untouched.
The result is a model trained on subtly incorrect ground truth. It learns the wrong associations between inputs and outputs, and those errors are baked into its weights before anyone notices something is wrong.
Think of it like this: you hand a student the correct exam answers to study, but someone quietly swapped a few correct answers for wrong ones before printing. The student studies hard, memorizes everything — and fails on exactly the questions that were swapped.
What makes label flipping particularly hard to spot in the context of data poisoning is the complete absence of data-level anomalies. The features of poisoned samples — pixel values, token distributions, numerical inputs — are entirely normal. Standard data validation checks look for corrupted files, out-of-range values, or unusual formats. None of those signals appear. According to arXiv survey (Goldblum 2021), flipped-label samples are indistinguishable from clean ones at the feature level, which is why they bypass the defensive checks teams typically run before training.
The foundational formulation came from adversarial machine learning research. According to Semantic Scholar (Xiao 2012), Xiao, Xiao, and Eckert (ECAI 2012) formulated label flipping as an optimization problem — not random label corruption, but a calculated strategy to maximize classification error. Earlier work by Biggio et al., documented by ResearchGate (Biggio 2011), showed that adversarially chosen flips cause disproportionately greater degradation than random labeling noise at the same poison fraction. Later research extended the technique to deep neural networks and NLP tasks, meaning no model architecture is inherently immune.
How It’s Used in Practice
Label flipping is a concern whenever training data is sourced from multiple parties, collected through public submissions, or managed through a federated setup. Each of those scenarios creates opportunities for adversarial label changes.
A common real-world example: a team builds a content moderation classifier trained on user-reported data. Users submit flagged examples with category labels. A bad actor systematically mislabels a subset — tagging harmful content as safe, or safe content as harmful — before the data is consolidated and fed into training. The model ships with a blind spot the team cannot easily trace back to the labeling phase.
In federated learning, the risk changes shape: individual participants train local models on their own data, and an adversary flips labels locally. Their corrupted local model contributes to the aggregated global model, and the attack propagates without any central visibility.
Pro Tip: Before training on any crowd-sourced or third-party dataset, check label distributions per class. Sudden shifts in class balance after data collection — especially shifts that favor a specific incorrect outcome — are one of the few statistical signals that can surface unsophisticated label flips before they reach training.
When to Use / When Not
| Scenario | Use ✅ | Avoid ❌ |
|---|---|---|
| Training on crowd-sourced or user-submitted labels | Audit label provenance and distributions | |
| Federated learning with untrusted participants | Validate each participant’s label distributions before aggregation | |
| Internal dataset labeled by a vetted in-house team | Access controls already reduce the attack surface | |
| Training data sourced exclusively from synthetic generation | Label flipping requires an adversary with write access to real labels | |
| Security-sensitive classifiers (spam, fraud, content safety) | Apply label sanitization before training | |
| Small, versioned datasets with full audit trails | Standard versioning and access logs provide lineage tracking |
Common Misconception
Myth: Label flipping is easy to detect because mislabeled examples will look unusual or out of place in the data.
Reality: The data features are completely normal. A legitimate email with its spam label flipped still looks like an ordinary email. Standard validation catches malformed inputs, out-of-range values, and corrupted files — not wrong labels. The attack is invisible to feature-level inspection. Only label-aware auditing methods, which compare labels against what the surrounding data suggests, can surface it.
One Sentence to Remember
Label flipping corrupts what the model is told to believe, not what it actually sees — which is why it slips past every standard data quality check that examines the data rather than the labels. If you work with training data from external sources, the labeling phase deserves the same scrutiny as the data features themselves.
FAQ
Q: How is label flipping different from random annotation errors? A: Random annotation errors are noise that affects accuracy uniformly and tends to cancel out at scale. Adversarial label flipping is strategic: labels are chosen to maximize specific classification failures, not simply introduce general noise.
Q: Can label flipping affect large language models? A: Yes, particularly during fine-tuning on instruction-label pairs or preference data. Flipping preference labels in RLHF datasets can steer a model toward harmful or incorrect outputs without altering the training text.
Q: How do you detect label flipping in a training dataset? A: According to arXiv (Label Sanitization), graph-based label sanitization methods — which use nearest-neighbor similarity to flag samples whose labels conflict with their local neighborhood — can detect and correct flipped labels before training starts.
Sources
- arXiv (Label Sanitization): Label Sanitization against Label Flipping Poisoning Attacks — Rosenfeld et al. (2018) — nearest-neighbor graph method for detecting and correcting flipped training labels
- OWASP: OWASP Top 10 for LLM Applications 2025 — LLM04: Data and Model Poisoning — official classification of label flipping under LLM data poisoning risks
Expert Takes
Label flipping is an adversarial optimization problem, not a data quality problem. An attacker doesn’t corrupt the input distribution — they corrupt the supervision signal. The model never sees “wrong” data; it sees correct data paired with wrong answers. This means representation-based anomaly detection won’t catch it. Detection requires label-aware methods that compare each sample’s label against what its nearest neighbors suggest it should be.
When building a training pipeline for any classifier that ingests third-party labels, add a label audit step before training starts. Track label provenance — which source contributed which labels — and flag classes where a single source dominates. In federated learning, treat each participant’s label distribution as untrusted until it passes a consistency check against global priors. Catching a label-flipping issue post-deployment costs far more than catching it before training.
The organizations that end up with label-flipped classifiers in production aren’t negligent — they’re trusting the wrong layer of their pipeline. Most ML security investment goes into model security: adversarial inputs, prompt injection, model inversion. The training data pipeline gets a fraction of that scrutiny. Label flipping exploits that gap. Teams that treat the labeling phase as a security boundary, not just a quality-control step, are ahead of where the field is now.
Label flipping raises a hard question about accountability. If a poisoned training set produces a model that systematically misclassifies a group of people — approving bad loans, flagging safe content — who is responsible? The attacker is anonymous. The developer didn’t know. The organization deployed in good faith. The harm is real and traceable to a deliberate act, but accountability dissolves at exactly the point where a label changed in a file no one thought to audit.