loading...
Training: log-loss against epoch (train solid, test dashed; grey line = constant predictor)
running...
How to use
Left: the applicant map. Each dot is a test applicant (circle = Group A, square = Group B; teal = repaid, orange = defaulted; a red ring marks an approved applicant who defaulted). The shaded field is the model's risk and the white line is the approval boundary. Hover a dot for its card. Right and below: ROC curve, score histograms, calibration and fairness bars.
Try this: (1) Press Restart training and watch the loss graph fall and the weights climb toward the true ones. (2) Click or drag in the histogram to move the threshold and watch profit, approvals and the red-ring defaulters change. (3) Preset Overfit tree: train AUC jumps, test AUC drops; run the depth sweep. (4) Preset Large historic gap, then Equalise approval rates: the approval bars match, profit falls and the repay rate of group B approvals drops. (5) Raise hidden risk and watch the ceiling AUC sink.
Drag to orbit, wheel to zoom. Space pauses training, R restarts it. Everything is synthetic; the group labels carry no real-world meaning.
The Blender 5.2 scene (committee room, plinth, glass gate, monitors, bins, fairness tubes) is baked to PBR textures and exported as one Draco GLB; the risk landscape, applicants and gauges are driven live from the same model as the 2D lab.
The science
Synthetic data, not real people. Every applicant is generated by the page. Four features (income, debt-to-income, card utilisation, credit history) are drawn as correlated standard normals; a hidden normal factor h adds risk that the model never sees. The true default probability is
p = σ(b0 + wtrue · x + sh h), σ(z) = 1 / (1 + e-z), wtrue = (-0.90, +0.95, +0.80, -0.55)
and the outcome is a coin flip with probability p. The intercept b0 is solved by bisection so that the average p equals the chosen base default rate. Group A and Group B are arbitrary labels. The group gap slider shifts the feature means of B (lower income and history, higher debt and utilisation) by a multiple of one standard deviation: the group never changes the true default probability directly, it only changes the typical applicant. Nothing here describes real borrowers or any real lender.
Logistic regression. The model predicts p̂ = σ(w · z + b) on standardised inputs z and minimises the regularised log-loss by full-batch gradient descent:
L(w) = -(1/n) Σ [ y ln p̂ + (1-y) ln(1-p̂) ] + (λ/2) ‖w‖², ∂L/∂w = (1/n) Σ (p̂ - y) z + λ w, w ← w - η ∂L/∂w
The loss is convex, so a small enough learning rate η converges to the single optimum; a rate above roughly 2/(largest curvature) makes the iterates overshoot and oscillate (try the unstable preset). Because the true model is logistic and well specified, with enough applicants and no hidden risk the learned weights approach wtrue (check 6). Decision tree. CART splits on the feature and cut that most reduces Gini impurity G = 2 q (1-q); a leaf predicts (defaults + 1)/(n + 2). Deep trees with tiny leaves memorise the training set, so train AUC rises while test AUC falls.
ROC and AUC. Sorting applicants by predicted risk and sweeping the cut-off traces the true-positive rate (defaulters caught) against the false-positive rate (good applicants declined). The AUC is the area under that curve and equals the probability that a random defaulter is ranked above a random repayer:
AUC = P(sdefault > srepaid) = (Σ ranks of defaulters - n1(n1+1)/2) / (n1 n0) (Mann-Whitney U)
The green dashed ceiling scores applicants with the true p, which no model can beat on this data: with hidden risk sh > 0 it is well below 1. Calibration. A calibrated model satisfies P(default | p̂ = q) = q. The reliability diagram bins test applicants by predicted risk (ten equal-count bins) and plots the observed default rate; ECE = Σ (nb/n) |mean p̂ - observed rate|, Brier = mean (p̂ - y)². AUC measures ranking only; calibration measures whether the numbers mean what they say. A model can rank well and still be badly calibrated.
Threshold and profit. Approve when p̂ < t. A repaid loan earns g, a default loses L, a declined applicant earns zero, so the expected profit of approving someone with true risk p is (1-p) g - p L, positive exactly when
p < t* = g / (g + L) (g = 1, L = 5 gives t* = 1/6 = 16.7 %)
The profit graph shows the model's curve against the same curve computed from the true probabilities. Fairness metrics. Demographic parity: equal approval rates in both groups. Equal opportunity: equal share of good (repaying) applicants approved in each group. Calibration within groups: the reliability curve is the same for A and B. When the groups have different base default rates, these criteria cannot all hold at once for an imperfect score (Kleinberg-Mullainathan-Raghavan; Chouldechova 2017): the group-specific thresholds that equalise approval rates break the single-threshold optimum and cost profit, and the page shows that cost instead of hiding it. Whether a given gap is acceptable is a legal and ethical question the simulation cannot answer.
Checks run in the page. (1) the trapezoid AUC equals the independent rank-sum AUC; (2) the analytic gradient matches a central finite difference; (3) the profit tally equals a direct sum over applicants; (4) the profit-maximising threshold for true probabilities is g/(g+L); (5) the true probabilities are calibrated on 3 600 fresh applicants; (6) gradient descent recovers wtrue on a fresh large sample with no hidden risk and no gap.
Simplifications. Four invented features, a linear-logistic truth, one time period, no interest-rate or term structure, no reject inference (real lenders never see outcomes of declined applicants; here the outcome is known for everyone), no regulation-specific rules, and the test set is a random 30 % of the same population, so there is no dataset shift. Real scorecards use many more variables, monotonic constraints, documented adverse-action reasons and legally defined protected attributes.