Algorithmic fairness: score distributions

Drag a gate along the score axis and watch selection, opportunity, false-positive and precision gaps compete.

Switch to 3D

Presets

Playback

Population and score

Thresholds (or drag the gates)

Fairness fixes

Layers

Model check

The science

What is modelled

A screening decision (a loan, a job interview, a medical follow-up) based on one numeric score. Every person is either truly qualified or not; the score is only a noisy hint. People are dots placed along the score axis and a gate decides at a threshold; the curves show the analytic score distributions the dots are drawn from, and the shaded tails are exactly the true-positive and false-positive rates. Because we know the hidden truth in this simulation we can count the four outcomes exactly: true positives (green), false negatives (red), false positives (amber) and true negatives (grey).

Equations

  • Score model: s ~ N(μ1, σ) for qualified people and s ~ N(μ0, σ) for the others; group B qualified people can have their score shifted down by the measurement bias δ (for example a test that under-measures their skills).
  • Rates at threshold t: TPR = 1 - Φ((t - μ1)/σ), FPR = 1 - Φ((t - μ0)/σ), selection rate = p TPR + (1 - p) FPR, precision PPV = p TPR / selection rate, where p is the share of qualified people (base rate).
  • Criteria: demographic parity (equal selection rates), equal opportunity (equal TPR), equalised odds (equal TPR and FPR), predictive parity (equal PPV). The numbers in the readouts are counted on the sampled people; the curves in the graphs are the analytic model.
  • Identity (Chouldechova 2017): FPR = p/(1 - p) · (1 - PPV)/PPV · TPR. It is why, when two groups have different base rates p, equal TPR, equal FPR and equal PPV cannot all hold at once unless the classifier is perfect.
  • Equal-opportunity threshold for B: tB = tA + (μ1,B - μ1,A); equal-selection threshold: solved by bisection on the analytic selection rate.

Checks run in the page

  • Sampled TPR against the analytic formula (within four standard errors).
  • The Chouldechova identity on both sampled confusion matrices.
  • The equal-opportunity threshold really equalises TPR, and what it costs in the other criteria.
  • The normal CDF against textbook values.

Honest simplifications

  • The population, base rates, score distributions and bias are invented illustrations; they are not data about any real group or system.
  • Real scores are rarely Gaussian, real labels are often themselves biased (here only the score is), and the right fairness criterion depends on the harm at stake and on law, not on a formula.
  • Group-specific thresholds can be legally restricted in some settings; they are used here only to show the trade-offs.
  • Samples of 50 to 400 people are noisy by design: press "New random people" to see how much the gaps move by chance alone.

What to try

  • "Equal groups, one threshold": all gaps are near zero (only sampling noise).
  • "Different base rates" with one threshold: selection and precision gaps appear even though the score is equally good for both groups.
  • Press "Equal opportunity (TPR)" and then look at the third graph: the TPR gap vanishes but the others do not.
ROC curves with operating points
Metrics by group
Gaps as group B threshold moves
Group A rates vs threshold
Made with 🟠 Blender · Three.js