Introduction

Every time a predictive algorithm sorts people — flagging loan applicants, recommending parole, screening résumés — someone asks: is it fair? The question sounds simple. The mathematics turns out to be brutal.

There are several natural ways to define "fair." A model might be calibrated: when it says "70% risk," roughly 70% of those flagged should actually be high-risk. Or it might equalise false-positive rates across groups: the fraction of innocent people wrongly flagged should be the same in every demographic. Or it might equalise false-negative rates: the fraction of actually-risky people who slip through undetected should be the same everywhere.

All three sound reasonable. All three are demanded by real institutions and legal frameworks. And in 2016–2017, two independent research teams — Chouldechova and Kleinberg, Mullainathan & Raghavan — proved that you generally cannot have all three at once. When two groups differ in their base rates (the true underlying prevalence of the outcome), satisfying any two of these criteria forces a violation of the third. This is not a limitation of today's algorithms. It is a mathematical theorem.

Try It: Move the Threshold

The two groups below have different base rates: Group A has a 30% true positive rate and Group B has a 60% true positive rate. Move the shared decision threshold and watch what happens to calibration, false-positive rate (FPR), and false-negative rate (FNR) for each group.

<!-- Demo: two groups with different base rates, shared threshold -->
<div class="demo-wrap">
  <p class="desc">Two groups share a classifier but differ in base rates (true prevalence). Move the threshold and observe how calibration (PPV), false-positive rate (FPR), and false-negative rate (FNR) change — and why you cannot equalise all three.</p>
  <div class="threshold-row">
    <label for="thresh">Threshold: <span id="thresh-val">0.50</span></label>
    <input type="range" id="thresh" min="0" max="100" value="50" step="1">
  </div>
  <div class="groups">
    <div class="group" id="group-a">
      <h3>Group A</h3>
      <p class="base-rate">Base rate: <strong>30%</strong></p>
      <div class="metrics" id="metrics-a"></div>
    </div>
    <div class="group" id="group-b">
      <h3>Group B</h3>
      <p class="base-rate">Base rate: <strong>60%</strong></p>
      <div class="metrics" id="metrics-b"></div>
    </div>
  </div>
  <div class="verdict" id="verdict"></div>
  <canvas id="chart" width="520" height="160" aria-label="Score distribution for Group A and Group B with threshold marker"></canvas>
  <p class="chart-caption">Score distributions — darker = truly positive, lighter = truly negative. Orange line = threshold.</p>
</div>

/* Layout for the fairness tradeoffs demo */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; padding: .5rem; }
.demo-wrap { max-width: 560px; margin: 0 auto; }
.desc { font-size: .88rem; color: #444; margin: 0 0 .8rem; line-height: 1.5; }
.threshold-row { display: flex; flex-direction: column; gap: .25rem; margin-bottom: .8rem; }
.threshold-row label { font-weight: 600; font-size: .9rem; }
#thresh { width: 100%; accent-color: #1d3557; }
.groups { display: grid; grid-template-columns: 1fr 1fr; gap: .8rem; margin-bottom: .7rem; }
.group { background: #f0f4f8; border-radius: 10px; padding: .7rem .8rem; }
.group h3 { margin: 0 0 .25rem; font-size: .95rem; color: #1d3557; }
.base-rate { margin: 0 0 .4rem; font-size: .82rem; color: #555; }
.metrics { display: flex; flex-direction: column; gap: .3rem; }
.metric-row { display: flex; justify-content: space-between; align-items: center; font-size: .83rem; }
.metric-label { color: #444; }
.metric-val { font-weight: 700; font-size: .9rem; }
.ok { color: #0a7d33; }
.warn { color: #c92f3c; }
.neutral { color: #1d3557; }
.verdict { font-size: .88rem; font-weight: 600; min-height: 1.6em; padding: .4rem .6rem;
           border-radius: 8px; margin-bottom: .6rem; background: #fff3cd; color: #6b4c00; }
canvas { display: block; width: 100%; border-radius: 8px; background: #f9fafb; border: 1px solid #dde3ea; }
.chart-caption { font-size: .78rem; color: #666; margin: .3rem 0 0; text-align: center; }

/* Fairness impossibility demo: calibration vs FPR vs FNR */
const i18nFmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));

/* Group base rates (true positive prevalence in each group) */
const BASE_A = 0.30; /* 30% true positives in Group A */
const BASE_B = 0.60; /* 60% true positives in Group B */
const N = 300;       /* simulated individuals per group */

/* Generate synthetic scores: positives cluster higher, negatives lower */
function lcg(seed) {
  /* deterministic LCG pseudo-random number generator */
  let s = (seed >>> 0);
  return function() { s = (s * 1664525 + 1013904223) & 0xffffffff; return (s >>> 0) / 0x100000000; };
}
function genScores(base, n, seed) {
  const rand = lcg(seed);
  const scores = [];
  for (let i = 0; i < n; i++) {
    const isPos = i < Math.round(n * base);
    /* Box-Muller transform: uniform pair -> standard normal sample */
    const u1 = Math.max(rand(), 1e-10), u2 = rand();
    const z = Math.sqrt(-2 * Math.log(u1)) * Math.cos(2 * Math.PI * u2);
    /* shift mean by label: positives high, negatives low */
    const mean = isPos ? 0.62 : 0.38;
    const score = Math.max(0, Math.min(1, mean + z * 0.18));
    scores.push({ score, label: isPos ? 1 : 0 });
  }
  return scores;
}

const scoresA = genScores(BASE_A, N, 42);
const scoresB = genScores(BASE_B, N, 137);

/* Compute PPV, FPR, FNR at a given threshold */
function computeMetrics(scores, thresh) {
  let tp = 0, fp = 0, tn = 0, fn = 0;
  for (const s of scores) {
    const pred = s.score >= thresh ? 1 : 0;
    if (pred === 1 && s.label === 1) tp++;
    else if (pred === 1 && s.label === 0) fp++;
    else if (pred === 0 && s.label === 0) tn++;
    else fn++;
  }
  const ppv  = (tp + fp) === 0 ? 1 : tp / (tp + fp); /* positive predictive value (calibration) */
  const fpr  = (fp + tn) === 0 ? 0 : fp / (fp + tn); /* false-positive rate */
  const fnr  = (fn + tp) === 0 ? 0 : fn / (fn + tp); /* false-negative rate */
  return { ppv, fpr, fnr };
}

/* Render metric values into a group panel */
function pct(v) { return (v * 100).toFixed(1) + '%'; }

function renderMetrics(id, m) {
  const el = document.getElementById(id);
  el.innerHTML = `
    <div class="metric-row">
      <span class="metric-label">PPV (calibration)</span>
      <span class="metric-val neutral">${pct(m.ppv)}</span>
    </div>
    <div class="metric-row">
      <span class="metric-label">FPR</span>
      <span class="metric-val neutral">${pct(m.fpr)}</span>
    </div>
    <div class="metric-row">
      <span class="metric-label">FNR</span>
      <span class="metric-val neutral">${pct(m.fnr)}</span>
    </div>`;
}

/* Show which fairness criteria are (approximately) satisfied */
function renderVerdict(mA, mB) {
  const TOL = 0.05;
  const ppvOk = Math.abs(mA.ppv - mB.ppv) <= TOL;
  const fprOk = Math.abs(mA.fpr - mB.fpr) <= TOL;
  const fnrOk = Math.abs(mA.fnr - mB.fnr) <= TOL;
  const ok = [ppvOk && 'calibration ✓', fprOk && 'FPR ✓', fnrOk && 'FNR ✓'].filter(Boolean);
  const fail = [!ppvOk && 'calibration ✗', !fprOk && 'FPR ✗', !fnrOk && 'FNR ✗'].filter(Boolean);
  const okStr  = ok.length   ? i18nFmt('Satisfied (≈): %items%',   { items: ok.join(', ') })   : '';
  const failStr = fail.length ? i18nFmt('Violated: %items%', { items: fail.join(', ') }) : '';
  document.getElementById('verdict').textContent = [okStr, failStr].filter(Boolean).join('  ');
}

/* Draw score distribution bars on canvas with threshold line */
function drawChart(thresh) {
  const canvas = document.getElementById('chart');
  const ctx = canvas.getContext('2d');
  const W = canvas.width, H = canvas.height;
  ctx.clearRect(0, 0, W, H);

  const BAR_H = 12, GAP = 8, TOP = 30;
  const groups = [
    { scores: scoresA, label: 'Group A (base 30%)', color: '#1d3557', yOff: 0 },
    { scores: scoresB, label: 'Group B (base 60%)', color: '#e63946', yOff: BAR_H + GAP }
  ];

  /* draw score dots as thin vertical bars per individual */
  for (const g of groups) {
    const y = TOP + g.yOff;
    ctx.fillStyle = '#dde3ea';
    ctx.fillRect(20, y, W - 40, BAR_H);
    for (const s of g.scores) {
      const x = 20 + s.score * (W - 40);
      ctx.fillStyle = s.label === 1 ? g.color : (g.color + '60');
      ctx.fillRect(x - 1, y, 2, BAR_H);
    }
    ctx.fillStyle = '#222';
    ctx.font = '11px system-ui, sans-serif';
    ctx.fillText(g.label, 20, y - 4);
  }

  /* draw threshold line in orange */
  const tx = 20 + thresh * (W - 40);
  ctx.strokeStyle = '#f4a261';
  ctx.lineWidth = 2;
  ctx.setLineDash([4, 3]);
  ctx.beginPath();
  ctx.moveTo(tx, TOP - 6);
  ctx.lineTo(tx, TOP + (BAR_H + GAP) + BAR_H + 6);
  ctx.stroke();
  ctx.setLineDash([]);

  ctx.fillStyle = '#f4a261';
  ctx.font = 'bold 11px system-ui, sans-serif';
  ctx.fillText('t = ' + (thresh * 100).toFixed(0) + '%', Math.max(tx - 10, 20), TOP - 10);

  /* draw colour legend */
  const ly = TOP + (BAR_H + GAP) + BAR_H + 22;
  ctx.font = '10px system-ui, sans-serif';
  ctx.fillStyle = '#1d3557'; ctx.fillRect(20, ly, 10, 10);
  ctx.fillStyle = '#222'; ctx.fillText('A pos', 34, ly + 9);
  ctx.fillStyle = '#1d355760'; ctx.fillRect(140, ly, 10, 10);
  ctx.fillStyle = '#222'; ctx.fillText('A neg', 154, ly + 9);
  ctx.fillStyle = '#e63946'; ctx.fillRect(260, ly, 10, 10);
  ctx.fillStyle = '#222'; ctx.fillText('B pos', 274, ly + 9);
  ctx.fillStyle = '#e6394660'; ctx.fillRect(380, ly, 10, 10);
  ctx.fillStyle = '#222'; ctx.fillText('B neg', 394, ly + 9);
}

function update() {
  const raw = parseInt(document.getElementById('thresh').value, 10);
  const thresh = raw / 100;
  document.getElementById('thresh-val').textContent = thresh.toFixed(2);
  const mA = computeMetrics(scoresA, thresh);
  const mB = computeMetrics(scoresB, thresh);
  renderMetrics('metrics-a', mA);
  renderMetrics('metrics-b', mB);
  renderVerdict(mA, mB);
  drawChart(thresh);
}

document.getElementById('thresh').addEventListener('input', update);
update(); /* initial render */
Result
Console

No matter where you place the threshold, at least one fairness criterion breaks. Lower the threshold and you equalise FNR — but FPR diverges. Raise it and FPR equalises — but FNR diverges. The groups are calibrated by construction, but that calibration coexists with persistent gaps in error rates. The impossibility is not a bug in the slider — it is a theorem.

The Real Complexity

Let two groups have prevalences (base rates) pAp_A and pBp_B with pA≠pBp_A \ne p_B. A binary classifier assigns a score; we apply a threshold to get a binary decision. Define for each group:

  • PPV (positive predictive value / calibration): among those flagged positive, the fraction that are truly positive.
  • FPR (false-positive rate): among truly negatives, the fraction wrongly flagged positive.
  • FNR (false-negative rate): among truly positives, the fraction wrongly missed.

These three quantities are linked by a simple identity. If PPV is the same across groups and FPR is the same across groups, then FNR must differ — and vice-versa. The algebra is exact:

PPV=p⋅(1−FNR)p⋅(1−FNR)+(1−p)⋅FPR\text{PPV} = \frac{p \cdot (1 - \text{FNR})}{p \cdot (1 - \text{FNR}) + (1 - p) \cdot \text{FPR}}

Fix PPV and FPR to be equal across groups with pA≠pBp_A \ne p_B; solve for FNR — you get different values. There is no escape. Chouldechova (2017) derived this identity and showed it means that calibration and equalised error rates are jointly impossible when base rates differ. Kleinberg, Mullainathan & Raghavan (2016) proved the same constraint from a different angle using score distributions.

This is a proved impossibility result — a cousin of impossibility theorems in computational complexity. No amount of clever engineering lifts it. Society must choose which notion of fairness to prioritise for any given deployment, and that choice is inherently political.

Where It Matters

The impossibility theorem is not abstract. Every high-stakes classifier lives inside it:

  • Criminal justice: the COMPAS recidivism tool was shown by ProPublica (2016) to have higher FPR for Black defendants. Northpointe countered that COMPAS is calibrated. Both were right — the impossibility explains why both findings coexist.
  • Hiring: résumé-screening algorithms trained on historical hiring data inherit historical base-rate differences. Equalising callback rates by group (equalising FPR) conflicts with calibrated score interpretation.
  • Medical triage: a sepsis-prediction model calibrated on a diverse population can still have unequal FNR across demographic groups if prevalence differs — missing higher-risk patients in lower-prevalence groups.
  • Credit scoring: lenders regulated to have equal approval rates (equalised FPR) will necessarily have different default prediction accuracy (PPV) across groups with different default rates.

Understanding the theorem tells practitioners what to expect and forces a conscious, auditable choice among fairness criteria rather than an accidental one. It connects directly to ideas in PAC learning: what a model can and cannot guarantee from data.

Conclusion

Fairness in machine learning is not a dial you can simply turn up. Chouldechova and Kleinberg et al. showed that calibration, equal false-positive rates, and equal false-negative rates form an impossible trinity whenever the groups being compared have different base rates — which is nearly always.

This does not mean we should give up on fairness. It means we must be precise about which fairness we are pursuing, transparent about what we are sacrificing, and humble about the limits of what any algorithm can deliver. The theorem turns a vague aspiration into an exact constraint — and that is the first step toward genuine accountability.

Like P vs NP, the fairness impossibility draws a hard line around what is achievable. Unlike P vs NP, we already have the proof.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/fair-machine-learning-tradeoffs/Content licensed under CC BY-NC 4.0.