Defending Training Pipelines Against Data Poisoning with ART, Data Provenance, and ML-BOM in 2026

Your model hit 94% accuracy on the eval set. Production outputs started drifting three weeks later. You’ve checked the architecture, tuned hyperparameters, inspected the loss curves. Everything looks clean. The problem isn’t your model. The problem is what you fed it.

Data Poisoning works by corrupting training data before it reaches your pipeline. Not a firewall exploit — no intrusion alert fires. The bad samples look legitimate. The model learns from them anyway. By the time you notice the behavioral shift, you’ve shipped the compromised model and the poisoned dataset is three versions back in your history.

This guide builds the specification framework to catch that attack before the training run starts — and document what you shipped if it already happened.

Before You Start

You’ll need:

• Python 3.10, 3.11, or 3.12 (ART 1.20.1 requirement, per PyPI)
• The Adversarial Robustness Toolbox installed: pip install adversarial-robustness-toolbox
• A dataset with known sources — even rough provenance metadata helps
• Understanding of Backdoor Attack, Clean Label Attack, and Label Flipping attack types
• Familiarity with Data Leakage as a threat category

This guide teaches you: How to structure a defense-in-depth pipeline that blocks poisoned data at ingestion, detects what slips through at training, and produces a machine-readable audit record of everything that entered your model.

What Happens When You Skip the Defense Layer

You pull a public dataset from a repository. Add some scraped examples. Feed the whole batch to your training script. Three months later, a specific phrase in user input triggers a consistent, wrong output — every time, for every user. You have a backdoor attack in production.

This isn’t hypothetical. It’s the attack surface that makes Nightshade-style techniques dangerous at scale: poisoned samples look identical to clean ones. The model trains normally. Evals pass. The trigger activates only in deployment.

The same pipeline failure causes Label Flipping to go undetected — a few hundred mislabeled examples shifting decision boundaries imperceptibly — and RAG Poisoning to propagate through retrieval-augmented systems where no one’s checking document provenance before indexing.

Step 1: Specify Your Data Sources and Provenance Requirements

Before any sample enters training, you need to know where it came from. That’s not a philosophical position — it’s the specification step that makes every downstream defense work.

Your data source taxonomy has these parts:

• First-party data — generated, labeled, and stored by your team; full chain of custody; lowest risk
• Licensed third-party datasets — formal provenance exists; check terms and version locking
• Crawled or scraped data — provenance is partial at best; highest poisoning risk surface
• Synthetic data — provenance is the generation process itself; document the generator version and seed

For each source category, your ingestion spec must answer: who contributed this data, under what terms, and how do you verify that answer computationally?

Provenance specification checklist:

• Dataset origin URL or system identifier logged per sample or per batch
• Contributor identity or organization recorded (for third-party sources)
• Ingestion timestamp and pipeline version stamped on each batch
• License and consent status attached as metadata (not just in a README)
• Hash of the raw input stored before any preprocessing

How to Use Croissant 1.1 for Machine-Actionable Provenance

Croissant 1.1 (released February 2026, maintained by MLCommons with 30+ contributing organizations) is the dataset metadata standard that makes provenance computationally verifiable, not just documented. Hugging Face, Kaggle, Google Dataset Search, and OpenML all support it natively (Croissant Spec).

A Croissant record for your dataset captures: source URLs per split, structured usage policies (consent and licensing), governance tags, and a vocabulary framework that links to external ontologies. The “machine-actionable provenance” feature in v1.1 means your pipeline can validate this record programmatically before a single sample touches your training code.

Your ingestion gate spec:

• Require a valid Croissant 1.1 metadata file for every external dataset
• Reject datasets where the consentType field conflicts with your use case
• Lock dataset versions using the content-addressable hash in the Croissant record
• Flag crawled or scraped sources for additional ART scanning in Step 2

The Spec Test: If your ingestion pipeline can’t answer “where did sample 47,293 come from?” after training completes, you have no defense against supply-chain poisoning — and no audit trail for your EU AI Act documentation.

Step 2: Run ART Poisoning Detectors on Ingested Data

Provenance gates handle known-bad sources. ART handles unknown-bad samples that slip through legitimate-looking sources.

ART 1.20.1 (MIT license, Linux Foundation AI & Data Foundation governance, verified on ART’s GitHub repository) ships four poisoning detectors under art.defences.detector.poison. Each one catches a different attack pattern. Run them in sequence — they are not redundant.

Your ART detector stack:

• ActivationDefence (art.defences.detector.poison.ActivationDefence) — clusters model activations to separate clean and poisoned samples, based on the Chen et al. 2018 method. Best for detecting backdoor triggers that cause activation-space separation.
• SpectralSignatureDefense (art.defences.detector.poison.SpectralSignatureDefense) — outlier scoring via spectral analysis of feature representations (Tran et al. 2018). Catches poisoned samples that are geometrically distant from the clean class distribution.
• RONIDefense (art.defences.detector.poison.RONIDefense) — Reject on Negative Impact; removes samples that degrade model accuracy when included. Straightforward but effective for label-flipping and targeted degradation attacks.
• ProvenanceDefense (art.defences.detector.poison.ProvenanceDefense) — tracks data source contributors to flag suspicious origins. Works alongside your Step 1 provenance records; treats unknown-origin samples as suspect.

Context checklist for your ART scan step:

• Framework specified: ART 1.20.1 supports TensorFlow v2 (TF1 support was removed in a recent release — any tutorial code using TF1 imports will fail), Keras, PyTorch, scikit-learn, XGBoost, LightGBM, and CatBoost (ART Docs)
• Detector selection mapped to your attack surface: ActivationDefence for backdoors, SpectralSignature for feature-space anomalies, RONI for accuracy-degrading samples
• Threshold calibration specified before the scan — the default thresholds are starting points, not production values
• Quarantine behavior defined: flagged samples go to a review queue, not automatic deletion
• Scan logged with sample IDs, detector name, score, and disposition

The Spec Test: If your ART scan produces a binary “clean/poisoned” list with no score metadata and no quarantine queue, you’re running detection but not defense — any false positive silently drops a legitimate training sample.

Note on clean-label attack detection: ART’s SpectralSignatureDefense catches some clean-label variants, but this attack class is intentionally designed to fool activation-based detectors. No single tool provides complete coverage. The defense-in-depth approach — provenance gates plus multiple detectors — reduces the attack surface; it does not eliminate it.

Security & compatibility notes:

• ART TF1 Breaking Change: TensorFlow v1 and MXNet support were removed in a recent ART release. Any existing pipeline code using TF1 imports (import tensorflow.compat.v1) will fail at runtime. Migrate to TF2 before integrating ART 1.20.1.
• Spring AI Vector Stores: If your pipeline includes a RAG retrieval layer, CVE-2026-40967 and CVE-2026-40978 expose filter-expression and document-ID injection vulnerabilities against vector stores. CVE-2026-40966 enables cross-tenant memory leakage via conversation IDs. Patch Spring AI before integrating RAG-based data augmentation.
• MCP Tool Integrations: Agentic ML pipelines using MCP are affected by CVE-2025-54136 (MCP Tool Poisoning — structural vulnerability in agent context). Validate tool call inputs before they touch training data.

Step 3: Sequence the Pipeline Defense Layers

Order matters. Running ART detection after preprocessing corrupts your provenance record. Building your ML-BOM at the end of training means it can’t document what you rejected.

Build order:

1. Provenance gate first — Croissant 1.1 metadata validation before any sample is parsed. Reject datasets that fail; quarantine datasets with partial provenance for manual review. This runs before your feature engineering or preprocessing step.
2. Preprocessing with hash logging — every transformation step (normalization, augmentation, deduplication) gets logged with input/output hashes. This is the audit trail that makes your ML-BOM useful later.
3. ART scan on preprocessed data — run all four detectors on the transformed dataset. Log scores per sample. Move flagged samples to quarantine with their detector scores attached.
4. Quarantine review gate — human or rule-based review of flagged samples before they are permanently excluded or reinstated. This step prevents automated false positives from silently distorting your Class Imbalance and Dataset Bias profile.
5. ML-BOM generation before training — snapshot the clean dataset state, all provenance records, and detector outputs into a CycloneDX ML-BOM. Training starts with a documented, versioned dataset.

For each pipeline step, your spec must define:

• What it receives as input (schema and validation rules)
• What it produces as output (schema and hash)
• What constitutes a failure condition (not just errors — silent data drops count)
• Where flagged items go (quarantine path, not /dev/null)

The Architect’s Rule: If any step in your pipeline can discard data silently without logging which samples were dropped and why, an attacker who controls that step’s input controls what your model learns.

Step 4: Build and Validate Your CycloneDX ML-BOM

The ML-BOM is not a compliance checkbox. It’s the audit artifact that makes your defense reproducible and your training runs verifiable.

CycloneDX introduced ML-BOM support in v1.5 (June 2023); fields have been stable since then. CycloneDX v1.7 (released late October 2025) added a root-level “Citations” element that declares where BOM data originated — build system, tool, or manual input — with “Attributed To” and “Process” fields (FOSSA Blog). That element is the provenance-of-provenance feature your pipeline needs.

Your ML-BOM must document:

• Datasets: origin, version hash, provenance method, ethical considerations, and consent status
• Training methodology: framework versions, ART scan configuration, detector thresholds used
• AI framework versions: TensorFlow/PyTorch version, ART version (1.20.1 from PyPI)
• Performance metrics: eval results on the clean post-scan dataset
• Licensing: per-dataset license terms

Validation checklist:

• Every dataset in the BOM has a matching Croissant 1.1 provenance record — failure looks like: dataset appears in training but has no BOM entry
• Every ART scan run appears in the BOM with its detector configuration — failure looks like: you can’t reproduce the scan because the threshold values weren’t logged
• The quarantine disposition for every flagged sample is recorded — failure looks like: “we removed some samples” with no list of which ones or why
• BOM version is locked to the training run ID — failure looks like: you update the BOM after training and lose the point-in-time state

One caution here: the CycloneDX tooling ecosystem for ML-BOM is less mature than the SBOM ecosystem as of early 2026 — the spec exists and is well-designed, but production-ready standalone ML-BOM CLI generators are limited (CycloneDX Docs). Plan for custom tooling to populate the BOM from your pipeline logs rather than expecting off-the-shelf tooling to do it automatically.

EU AI Act relevance: Article 11 technical documentation requirements for high-risk AI take effect August 2, 2026. Annex IV fields map onto CycloneDX ML-BOM entries (Frontiers). If your system qualifies as high-risk under the Act, a structured ML-BOM is your fastest path to Annex IV documentation.

Common Pitfalls

Pro Tip

The hardest part of this pipeline to specify is the quarantine review gate — specifically, who reviews flagged samples and by what criteria. If you leave it as “a human looks at it,” you’ll skip it under deadline pressure. Write the review spec as a checklist: what constitutes a reinstatement decision, what constitutes a permanent exclusion, and what requires escalation. Attach that checklist to your ML-BOM as a process document. The defense is only as strong as the weakest gate.

Frequently Asked Questions

Q: How to build a defense-in-depth pipeline against data poisoning step by step?

A: The sequence is: provenance gate (Croissant 1.1 validation before parsing) → preprocessing with hash logging → ART scan on preprocessed data (all four detectors) → quarantine review gate → ML-BOM snapshot → training. Each layer has a specific failure mode the next layer doesn’t cover. Skipping any layer creates a gap an attacker can exploit. The quarantine review gate is where most teams cut corners — specify it as a written checklist before you need it.

Q: How to use the Adversarial Robustness Toolbox to detect poisoning attacks?

A: Install ART 1.20.1 via pip install adversarial-robustness-toolbox (Python 3.10–3.12). Run ActivationDefence for backdoor detection, SpectralSignatureDefense for feature-space outliers, RONIDefense for accuracy-degrading samples, and ProvenanceDefense for suspicious-origin flagging. Log scores per sample with detector name and threshold used. One edge case many teams miss: ART’s base class art.defences.detector.poison.PoisonFilteringDefence lets you build custom detectors for domain-specific attack patterns — useful when your data distribution is narrow enough that generic detectors produce too many false positives.

Q: How to use data provenance and ML-BOM to secure a training dataset?

A: Croissant 1.1 provides machine-actionable provenance at the dataset level — source URLs, consent status, license terms, governance tags. CycloneDX ML-BOM (v1.5+, stable; v1.7 adds Citations element for provenance-of-provenance) captures the full training artifact audit trail. Use Croissant records as your ingestion gate criteria and feed their hashes into the ML-BOM’s dataset entries. The key operational detail: generate the BOM before training and lock it to the training run ID. A BOM generated after training can’t document what was rejected and is much weaker as an audit artifact.

Your Spec Artifact

By the end of this guide, you should have:

• A provenance requirements checklist — one entry per source category (first-party, licensed, crawled, synthetic) with ingestion gate criteria and rejection behavior
• An ART scan configuration spec — detector selection mapped to your attack surface, threshold calibration method, quarantine rules, and logging schema
• A CycloneDX ML-BOM template — dataset entries with Croissant 1.1 hash fields, ART scan results section, quarantine disposition log, and training run ID lock

Your Implementation Prompt

Use this prompt in Claude Code, Cursor, or Codex when building or reviewing your data pipeline. Paste it at the start of a new context and fill in the bracketed values before generating any pipeline code.

You are helping me build a data poisoning defense pipeline for a machine learning training system.

SYSTEM CONTEXT:
- ML framework: [TensorFlow v2 / PyTorch / scikit-learn]
- ART version: 1.20.1 (Python 3.10–3.12)
- Dataset sources: [list your source categories: first-party / licensed-third-party / crawled / synthetic]
- Data volume: [approximate sample count]
- Training frequency: [one-time / weekly / continuous]

PIPELINE SPEC — build in this order:

1. PROVENANCE GATE
   - Validate Croissant 1.1 metadata file for every external dataset
   - Rejection behavior: [hard reject / quarantine for review]
   - Fields to validate: consentType, source URL, dataset hash
   - Output: ingestion log with pass/fail per dataset and rejection reason

2. PREPROCESSING WITH HASH LOGGING
   - Input schema: [define your raw data schema]
   - Transformations to apply: [normalization / augmentation / deduplication]
   - Logging: input hash, output hash, transformation name, timestamp per batch
   - Failure condition: any transformation that drops samples without logging the sample IDs

3. ART SCAN
   - Detectors to run: ActivationDefence, SpectralSignatureDefense, RONIDefense, ProvenanceDefense
   - Threshold calibration source: [known-clean held-out set of N samples]
   - Output: per-sample score file with columns [sample_id, detector_name, score, threshold, disposition]
   - Quarantine path: [path or queue name for flagged samples]

4. QUARANTINE REVIEW GATE
   - Review criteria: [list reinstatement vs. exclusion vs. escalation criteria]
   - Disposition logging schema: [sample_id, reviewer, decision, reason, timestamp]
   - Class balance check: run after quarantine to verify [target class distribution]

5. ML-BOM GENERATION
   - Format: CycloneDX v1.7 ML-BOM
   - Dataset entries: include Croissant 1.1 hash, provenance method, consent status, license
   - ART scan section: detector configuration, threshold values, flagged sample count, quarantine count
   - Training run ID: [how you generate this — timestamp, git hash, experiment tracker ID]
   - Lock BOM before training starts

VALIDATION CRITERIA:
- Every dataset in BOM has matching Croissant record
- Every ART scan run has logged threshold config
- Every quarantine disposition is documented
- BOM timestamp precedes training start timestamp

Generate the pipeline spec as a set of typed Python dataclasses for each pipeline stage's input/output schema, then implement the logging scaffolding for each stage. Do not implement the ART detector calls themselves — stub those with the correct method signatures from art.defences.detector.poison.

Ship It

You now have a four-layer defense — provenance gates, ART detection, quarantine review, ML-BOM documentation — with a specific spec for each layer and a clear failure mode when any layer is skipped. The mental model is simple: poisoning attacks work because pipelines have no chain of custody. Build the chain of custody first, then add detection on top of it. Deploy safe, Max.

Deploy safe, Max.