Simpson's paradox: how a trend can reverse completely when you aggregate the data

What Simpson's paradox actually is

Simpson's paradox occurs when a trend that appears in every subgroup of data disappears — or reverses entirely — when the subgroups are combined. It is named after Edward Simpson (1951), though the effect was noted decades earlier by Karl Pearson and Udny Yule. Formally, it's possible for both of these to hold simultaneously:

It isn't a paradox of arithmetic — the math is entirely consistent — it's a paradox of intuition: we expect combining "better in every case" to give "better overall," and that intuition is simply wrong once the subgroup sizes are unbalanced.

The UC Berkeley admissions case

The most famous real-world example: in 1973, UC Berkeley's overall graduate admission rates showed men admitted at a significantly higher rate than women — a gap large enough to trigger a lawsuit. But when Bickel, Hammel & O'Connell (1975) broke the data down by department, women were admitted at a higher or equal rate in most individual departments.

The resolution: women disproportionately applied to competitive departments (like C, admitting ~35% overall) while men disproportionately applied to less competitive ones (like A and B, admitting >60%). Department choice was a confounder that made the aggregate figure actively misleading about departmental bias.

Confounders: the hidden third variable

The mechanism behind every Simpson's paradox is a confounding variable Z that influences both the "treatment" variable and the outcome, and is unevenly distributed across the comparison groups. In causal-diagram terms:

In the Berkeley case, Z = department, X = applicant sex, Y = admission. Department affected both which sex applied more (X) and the baseline admission rate (Y) — the textbook signature of confounding.

Which number should you trust?

There is no universal rule — it depends entirely on the causal structure, and getting it backwards can be actively harmful:

• Trust the disaggregated (per-group) result when the grouping variable is a genuine confounder — something that causally affects both the assignment and the outcome, like department choice above.
• Trust the aggregate result when the grouping variable is itself a consequence of the treatment (a "collider" or mediator) — conditioning on it can introduce bias that wasn't there in the raw data.

A minimal reversal in JavaScript

// Two groups, each showing A > B, but combined totals reverse the order
const groupSmall = { A: { success: 81, total: 87 }, B: { success: 234, total: 270 } };
const groupLarge = { A: { success: 192, total: 263 }, B: { success: 55, total: 80 } };

function rate(g) { return g.success / g.total; }

console.log("Small trial:", rate(groupSmall.A).toFixed(3), "vs", rate(groupSmall.B).toFixed(3));
console.log("Large trial:", rate(groupLarge.A).toFixed(3), "vs", rate(groupLarge.B).toFixed(3));

const combinedA = { success: groupSmall.A.success + groupLarge.A.success, total: groupSmall.A.total + groupLarge.A.total };
const combinedB = { success: groupSmall.B.success + groupLarge.B.success, total: groupSmall.B.total + groupLarge.B.total };
console.log("Combined:", rate(combinedA).toFixed(3), "vs", rate(combinedB).toFixed(3));
// A wins both subgroups individually, but B wins once combined —
// because A got most of its data from the "harder", lower-rate group

Detecting the paradox in your own data

Before trusting an aggregate comparison, check for a plausible confounder and stratify by it:

1. Compute the overall effect (aggregate rates for each group).
2. Identify candidate confounders — anything that plausibly affects both which group a data point falls into and the outcome.
3. Stratify by each candidate and recompute the effect within strata.
4. If the sign flips, you have Simpson's paradox — draw a causal diagram to decide whether the aggregate or stratified estimate answers your actual question.

Where Simpson's paradox shows up

• Clinical trials: a treatment can look worse overall while being better in every age or risk subgroup, if sicker patients disproportionately received it (a classic kidney stone treatment study shows exactly this).
• Vaccine efficacy during a pandemic: case rates by vaccination status can reverse over time purely because vaccinated and unvaccinated populations skew toward different age groups with very different baseline risk.
• University admissions and hiring: the Berkeley case, and its many descendants in workplace pay-gap and hiring-rate analyses.
• Sports statistics: a player can have a higher batting average than another in every season individually, yet a lower career average, if their at-bat counts differ wildly by season.
• A/B testing: a website variant can win in every traffic segment but lose overall if segment mix differs between test arms.