Simpson's Paradox

In 1951, statistician Edward H. Simpson described a striking puzzle: a trend present in every subgroup of a dataset can vanish — or outright reverse — when those subgroups are merged into a single table. The effect had been noticed earlier (Yule, 1903; Cohen, 1986), but Simpson's 1951 paper made it famous enough to carry his name.

The paradox is not a mathematical error. It is a warning about what aggregation does to information. When you combine groups of different sizes that carry a hidden third variable — a confounder — the raw totals can point in the opposite direction from the truth inside each group.

The classic example comes from real life. In 1973, UC Berkeley's graduate admissions appeared to discriminate against women: overall admission rates were lower for women than for men. But when researchers looked department by department, women were admitted at equal or higher rates in most departments. The paradox: women disproportionately applied to the most competitive departments, which dragged down their combined rate. The apparent bias vanished when the confounder — department selectivity — was controlled.

Simpson's Paradox is not a quirk of toy examples. It shows up in medical trials, criminal sentencing data, baseball statistics, and COVID-19 mortality figures. It is the reason "aggregate data" is rarely the last word in any serious analysis.

Two hospitals are testing the same treatment. Drag the sliders to change how many patients each hospital treats and what their success rates are. The blue bars show each hospital separately; the gray bar shows the combined total.

<p class="hint">{{hint}}</p>
<div class="hospitals">
  <div class="hospital">
    <div class="hosp-label">{{hosp_a}} <span class="tag severe">{{severe}}</span></div>
    <label>{{treat_patients}} <b id="nTA-val">300</b>
      <input type="range" id="nTA" min="10" max="500" value="300" step="10">
    </label>
    <label>{{treat_success}} <b id="rTA-val">30%</b>
      <input type="range" id="rTA" min="5" max="95" value="30" step="1">
    </label>
    <label>{{ctrl_patients}} <b id="nCA-val">100</b>
      <input type="range" id="nCA" min="10" max="500" value="100" step="10">
    </label>
    <label>{{ctrl_success}} <b id="rCA-val">20%</b>
      <input type="range" id="rCA" min="5" max="95" value="20" step="1">
    </label>
  </div>
  <div class="hospital">
    <div class="hosp-label">{{hosp_b}} <span class="tag mild">{{mild}}</span></div>
    <label>{{treat_patients}} <b id="nTB-val">30</b>
      <input type="range" id="nTB" min="10" max="500" value="30" step="10">
    </label>
    <label>{{treat_success}} <b id="rTB-val">90%</b>
      <input type="range" id="rTB" min="5" max="95" value="90" step="1">
    </label>
    <label>{{ctrl_patients}} <b id="nCB-val">90</b>
      <input type="range" id="nCB" min="10" max="500" value="90" step="10">
    </label>
    <label>{{ctrl_success}} <b id="rCB-val">70%</b>
      <input type="range" id="rCB" min="5" max="95" value="70" step="1">
    </label>
  </div>
</div>
<div class="chart-area">
  <div class="chart-col">
    <div class="bar-pair" id="bars-A"></div>
    <div class="col-label">{{hosp_a}}</div>
  </div>
  <div class="chart-col">
    <div class="bar-pair" id="bars-B"></div>
    <div class="col-label">{{hosp_b}}</div>
  </div>
  <div class="chart-col combined">
    <div class="bar-pair" id="bars-C"></div>
    <div class="col-label">{{combined}}</div>
  </div>
</div>
<div class="legend"><span class="dot treat"></span>{{legend_treat}} &nbsp; <span class="dot ctrl"></span>{{legend_ctrl}}</div>
<div id="verdict" class="verdict"></div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; font-size: 14px; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .8rem; line-height: 1.45; }
.hospitals { display: flex; gap: .7rem; margin-bottom: .9rem; flex-wrap: wrap; }
.hospital { flex: 1 1 210px; background: #f3f6f9; border: 1px solid #dde3ea; border-radius: 10px; padding: .65rem .8rem; }
.hosp-label { font-weight: 700; margin-bottom: .45rem; display: flex; align-items: center; gap: .4rem; }
.tag { font-size: .72rem; font-weight: 600; border-radius: 4px; padding: 1px 6px; }
.tag.severe { background: #fde8e8; color: #b91c1c; }
.tag.mild { background: #e8f5e9; color: #166534; }
.hospital label { display: flex; align-items: center; gap: .4rem; font-size: .82rem; margin-bottom: .3rem; flex-wrap: wrap; }
input[type=range] { flex: 1 1 80px; accent-color: #1d3557; }
.chart-area { display: flex; gap: .6rem; align-items: flex-end; margin: .5rem 0 .25rem; height: 150px; }
.chart-col { display: flex; flex-direction: column; align-items: center; flex: 1; }
.chart-col.combined .bar-pair { border-left: 2px dashed #aaa; padding-left: 4px; }
.bar-pair { display: flex; gap: 4px; align-items: flex-end; height: 130px; width: 100%; justify-content: center; }
.bar { width: 38%; border-radius: 5px 5px 0 0; position: relative; min-height: 4px; transition: height .2s; }
.bar.treat { background: #1d7abd; }
.bar.ctrl  { background: #9fb2c8; }
.bar .pct  { position: absolute; top: -18px; left: 50%; transform: translateX(-50%); font-size: .7rem; font-weight: 700; white-space: nowrap; color: #222; }
.col-label { font-size: .78rem; font-weight: 600; margin-top: .3rem; color: #333; text-align: center; }
.legend { font-size: .8rem; display: flex; align-items: center; gap: .25rem; margin: .1rem 0 .4rem; }
.dot { display: inline-block; width: 10px; height: 10px; border-radius: 2px; }
.dot.treat { background: #1d7abd; }
.dot.ctrl  { background: #9fb2c8; }
.verdict { font-size: .88rem; font-weight: 600; border-radius: 8px; padding: .4rem .7rem; min-height: 2em; }
.verdict.paradox { background: #fef3c7; color: #92400e; border: 1px solid #fbbf24; }
.verdict.normal  { background: #e8f5e9; color: #166534; border: 1px solid #86efac; }

// Code not found

Notice: you can arrange things so both Hospital A and Hospital B have a higher success rate for the treatment than for the control — yet the combined bar shows the opposite. That reversal is Simpson's Paradox. The key is that one hospital handles far more severe cases than the other, so the mix of patient types in the aggregate drowns out the within-group signal.

Simpson's Paradox has a clean algebraic explanation. Suppose group A has success rate $a_{1}$/$b_{1}$ and group B has success rate $a_{2}$/$b_{2}$, with $a_{1}$/$b_{1}$ > $a_{2}$/$b_{2}$ in both subgroups of every stratum. But when you pool the two groups:

($a_{1}$ + $a_{2}$) / ($b_{1}$ + $b_{2}$) is a weighted average of the two rates, with weights $b_{1}$ and $b_{2}$. If the group with the lower rate is much larger, the weighted average can fall below the rate for group A in every stratum — even though group A won in each one individually.

The deeper issue is causality, not arithmetic. Statisticians Judea Pearl and others showed in the 1990s that Simpson's Paradox always signals a confounding variable — a hidden factor that influences which group a subject falls into and the outcome. In the Berkeley case: department selectivity. In clinical trials: disease severity. Without accounting for the confounder, the aggregate statistic is not just imprecise — it can give the wrong sign.

The resolution is never automatic. Sometimes the subgroup analysis is correct; sometimes the aggregate is. Which one to trust depends on the causal structure of the problem, not on the numbers alone. This is why Bayesian inference and causal graphs have become essential tools in statistics: they force you to make the causal model explicit before reading off a conclusion.

Simpson's Paradox is proven to occur whenever a confounder satisfies two conditions simultaneously: (1) it correlates with group membership, and (2) it correlates with the outcome. Both conditions are disturbingly common in observational data.

Simpson's Paradox is not a classroom curiosity — it has reversed real decisions in high-stakes settings:

• Medical trials: A treatment can look worse overall if sick patients are routed to it more often. Drug approvals and meta-analyses must stratify by disease severity or risk reversing the correct conclusion.
• Berkeley admissions (1973): The famous gender-bias study that turned out to be an artifact of department-level confounding — one of the most cited examples in statistics.
• COVID-19 mortality: Early in the pandemic, some countries appeared to have lower case-fatality rates for older patients than younger ones in raw data — until researchers realized the age distribution of tested patients differed drastically between countries.
• Baseball batting averages: David Justice had a higher batting average than Derek Jeter in both 1995 and 1996, yet Jeter's combined average for those two years was higher. The paradox arose because their at-bat counts were very different across years.
• Criminal sentencing: Studies of racial disparities in sentencing can reverse direction when controlled for crime type, prior record, and jurisdiction — making causal claims from aggregate data legally and ethically dangerous.

The lesson is not that data lies, but that aggregation without controlling for confounders is a form of measurement error. Understanding Simpson's Paradox is a prerequisite for any serious work in Bayesian inference or data analysis.

Simpson's Paradox is one of statistics' most uncomfortable truths: a trend you trust in every subgroup can reverse the moment you combine the groups, simply because a hidden variable distorts the mix.

The fix is not more data. It is the right question: what caused the groups to differ in size? Once you draw the causal graph and identify the confounder, you know which analysis to trust — the stratified one or the aggregate one. Without that causal reasoning, even a perfectly clean dataset can send you in exactly the wrong direction.

In a world drowning in dashboards and summary statistics, Simpson's Paradox is a quiet reminder that numbers never speak for themselves. The story behind the grouping always matters more than the totals.