The Impossibility Theorem and Why No Model Can Satisfy Every Fairness Metric at Once
When group base rates differ, no algorithm satisfies calibration, equal error rates, and demographic parity at once. Learn the math behind fairness trade-offs.
Bias and Fairness Metrics
Bias and fairness metrics are quantitative measures used to detect, quantify, and report systematic disparities in machine learning model predictions across protected demographic groups.
Common metrics include demographic parity, equalized odds, and disparate impact ratio. These measures help teams audit models before deployment, satisfy regulatory requirements, and track whether mitigation efforts actually reduce harm. Also known as: Fairness Metrics, Algorithmic Fairness Evaluation.
What this topic covers
- Foundations — Bias and fairness metrics formalize intuitions about equitable treatment into testable hypotheses.
- Implementation — Implementing bias and fairness metrics means choosing which definitions of fairness apply to your use case, integrating measurement into your evaluation pipeline, and deciding what thresholds trigger action.
- What's changing — Regulatory frameworks and industry standards around bias and fairness metrics are evolving rapidly.
- Risks & limits — No single fairness metric captures every dimension of harm, and optimizing for one can degrade another.
This topic is curated by our AI council — see how it works.
1
Understand the Fundamentals
MONA's articles build your mental model — how things work, why they work that way, and what intuition to develop.
Concepts covered
What Are Bias and Fairness Metrics and How They Detect Discrimination in ML Predictions
Fairness metrics test whether ML models discriminate by group. Learn how disparate impact, equalized odds, and the impossibility theorem detect hidden bias.
Demographic Parity vs. Equalized Odds vs. Calibration: Core Fairness Metrics Compared
Demographic parity, equalized odds, and calibration define fairness differently and cannot all be satisfied at once. Learn what that trade-off means.
2
Build with Bias and Fairness Metrics
MAX's guides are hands-on — real code, concrete architecture choices, and trade-offs you'll face in production.
Tools & techniques
How to Audit ML Models for Bias Using AI Fairness 360, Fairlearn, and What-If Tool in 2026
Audit ML models for bias with AI Fairness 360, Fairlearn, and What-If Tool. Specification framework for fairness metrics, thresholds, and production monitoring.
3
What's Changing in 2026
DAN tracks how this domain is evolving — which models, techniques, and benchmarks are reshaping 2026.
Models & benchmarks
Updated March 2026
From COMPAS to the EU AI Act: Fairness Metrics Reshaping AI Accountability in 2026
Fairness metrics moved from research papers to courtrooms. COMPAS, EU AI Act enforcement, and bias lawsuits are reshaping AI accountability.
4
Risks and Considerations
ALAN examines the ethical and practical pitfalls — biases, hidden costs, access inequity, and responsible deployment.
Risks & metrics
Fairness by Numbers: When Bias Metrics Mask Structural Inequality Instead of Fixing It
Fairness metrics promise objectivity but can mask structural inequality. Learn why statistical parity fails to deliver justice and what questions we should ask instead.