AI Diagnostic Threshold Lab

Classifier Controls

Decision threshold 3.00

Test discrimination (mean separation) 4.0

Score noise (spread σ) 1.20

Disease prevalence 10%

Confusion Matrix (n=300)

	Predicted +	Predicted −
Actually diseased	0	0
Actually healthy	0	0

Sensitivity (TPR)

Specificity (TNR)

PPV (precision)

NPV

Accuracy

AUC

0.00

How it works

Each simulated patient has an AI-model "risk score". Diseased patients' scores are drawn from N(separation, σ), healthy patients' from N(0, σ). The classifier calls a patient positive whenever score ≥ threshold:

Sensitivity (TPR) = TP / (TP + FN)
Specificity (TNR) = TN / (TN + FP)
Youden's J        = Sensitivity + Specificity − 1  (maximised at the optimal cutoff)
AUC               = ∫ TPR d(FPR)  over every possible threshold (trapezoidal rule)

Sensitivity and specificity depend only on the two score distributions, not on how common the disease is. Predictive values do — a positive result means much less when disease is rare, via Bayes' rule:

PPV = (Sens·Prev) / (Sens·Prev + (1−Spec)·(1−Prev))
NPV = (Spec·(1−Prev)) / (Spec·(1−Prev) + (1−Sens)·Prev)

Threshold — slide the cutoff; the plane sweeps through the 3D cloud and the confusion-matrix cells recount instantly.
Discrimination — how far apart the two populations' true means sit; a better AI model separates them more, raising AUC.
Noise — how much the two clouds overlap; more overlap makes any single threshold a worse trade-off.
Prevalence — same sensitivity/specificity, wildly different PPV in a rare-disease screening setting vs a high-risk clinic.
Snap to Youden optimum — moves the threshold to the point that maximises Sensitivity + Specificity − 1, the classic "best single cutoff".

Real-world relevance: this is exactly the trade-off a radiologist AI, a sepsis-alert model, or a cancer-screening test faces — the same underlying model can be "tuned" toward catching more true cases (high sensitivity, more false alarms) or toward fewer false alarms (high specificity, more misses), and the ROC curve is how regulators and clinicians compare AI diagnostic tools objectively.

Diseased patient Healthy patient Decision threshold plane

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.