AI Teacher Fairness & Calibration Lab

AI Grading Model

AI bias against Group B +12 pts

Debiasing correction 0%

Pass threshold 60

Students per group 60

Legend

Group A — passing Group A — failing Group B — passing Group B — failing

Fairness Metrics

Demographic parity gap

0.0 pp

Calibration error (MAE)

0.0 pts

Overall pass rate

0.0%

How it works

Each dot is one simulated student, placed by true ability (horizontal axis) against the AI teacher's predicted score (vertical axis). A perfectly fair, perfectly calibrated model would place every student exactly on the white diagonal reference line — predicted score equals true ability, for every student, regardless of group.

raw(s)       = ability(s) + bias · g(s) + noise(s)
corrected(s) = raw(s) − k · bias · g(s)
pass(s)      = corrected(s) ≥ threshold

where g(s) is +1 for Group B and 0 for Group A, bias is the AI bias slider, k is the debiasing-correction strength (0–1), and noise(s) is a small fixed per-student measurement error.

AI bias against Group B — a positive value means the raw model systematically over- or under-scores Group B relative to their true ability, independent of how capable they actually are. This is a stand-in for real documented failure modes: an essay-grading model trained mostly on one dialect, or a plagiarism detector that flags non-native phrasing more often.
Debiasing correction — a post-hoc fairness adjustment (the kind used in real ML pipelines, e.g. equalized-odds post-processing) that subtracts a fraction of the known group bias back out. At 100% the bias is fully cancelled and the two groups' clouds re-align.
Demographic parity gap — |pass-rate(A) − pass-rate(B)|, the headline fairness metric regulators and researchers use to flag disparate impact.
Calibration error — mean |corrected score − true ability| across every student: how far the AI teacher's assessment drifts from reality, bias aside.

The point of the simulator: a model can be highly accurate on average and still be badly unfair to one group — accuracy and fairness are measured separately, and an AI teacher deployed without checking both can quietly misjudge thousands of students.

Perfect calibration (y = x) Pass threshold plane

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.