Base-Rate Fairness Impossibility: Why Equal Precision and Equal Error Rates Cannot Both Hold
Two groups with different base rates meet the same binary classifier. Enforce either calibration (equal precision and sensitivity) or equalised odds (equal error rates) and watch the other criterion break by an amount fixed by Bayes rule and the base-rate odds. Checked against raw confusion counts, the identity round trip and a Monte Carlo population. A Blender-built data bench with the confusion cells of both groups.
A machine learning classifier scores two demographic groups using exactly the same underlying risk model — yet the groups have different true base rates for the outcome being predicted. This simulator renders both populations as 3D point clouds along a shared score axis and lets you set each group's base rate and the decision threshold(s), then watches equalized-odds metrics (FPR, TPR) and the calibration metric (PPV) update live. The result reproduces a real mathematical fact known as the fairness impossibility theorem: when base rates differ, equal accuracy across groups and equal calibration across groups cannot both hold at once, except in the trivial case where the base rates already match.
Two groups with different base rates meet the same binary classifier. Enforce either calibration (equal precision and sensitivity) or equalised odds (equal error rates) and watch the other criterion break by an amount fixed by Bayes rule and the base-rate odds. Checked against raw confusion counts, the identity round trip and a Monte Carlo population. A Blender-built data bench with the confusion cells of both groups.
3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install