Model Evaluation Simulator

Classifier scores

Samples 1200

Class separation 0.55

Score spread (noise) 0.16

Positive class prior 0.35

Decision threshold

Threshold t 0.50

–

Precision

–

Recall (TPR)

–

FPR

–

AUC-ROC

–

AUC-PR (AP)

–

How it works

Every point below is a synthetic sample drawn from one of two score distributions — negatives centered lower, positives centered higher, overlapping by an amount you control. A real classifier outputs exactly this: a continuous score per item, not a hard label. The hard label only appears once you pick a decision threshold t and declare "predict positive" wherever score ≥ t.

TPR (Recall) = TP / (TP + FN)
FPR = FP / (FP + TN)
Precision = TP / (TP + FP)
AUC = Σ trapezoid area under the swept curve

ROC curve — plots TPR against FPR as t sweeps from 1 to 0. A perfect separator hugs the top-left corner; a random classifier tracks the diagonal. AUC-ROC is the probability a random positive outscores a random negative.
Precision-Recall curve — plots Precision against Recall. It reacts to class imbalance where ROC does not: with few positives, small FP counts still crater precision, so PR is the sharper diagnostic on skewed data.
Class separation / spread — widen the gap between the two distributions or shrink their spread and both curves bow toward the perfect corner; overlap them and both curves collapse toward their baselines.
Positive prior — lowering it makes positives rarer. ROC barely moves (it's threshold-symmetric per class) but the PR curve's baseline drops with it — proof the two metrics answer different questions.

Real-world relevance: choosing where to sit on this curve — not just which model wins on AUC — is the actual decision behind a fraud filter, a spam gate or a medical screening test, where false positives and false negatives cost very different amounts.

Negative scores Positive scores Threshold t