📊 The Class-Imbalance Trap

About this simulation

Each glowing point is a simulated patient placed in a two-feature diagnostic space (e.g. a biomarker score on one axis, a secondary marker on the other). Most patients are healthy (majority class); only a small fraction actually carry the rare disease (minority class), and their biomarkers overlap with the healthy population — exactly as happens in real rare-disease screening. A translucent decision plane is the classifier: everything to its right is flagged "at risk."

How it works

• True positives (bright red) — disease correctly caught by the classifier.
• False negatives (purple) — disease the model missed. In medicine, these are the dangerous errors.
• False positives (yellow) — healthy patients incorrectly flagged.
• True negatives (teal) — healthy patients correctly cleared.
• Accuracy counts all correct calls, so with a rare disease it can stay high even while the model misses almost every real case.
• F1-score balances precision and recall, so it collapses the moment the model stops finding positives — exposing what accuracy hides.

Controls

• Disease prevalence — how rare the condition is in the simulated population (1–40%).
• Population size — number of simulated patients.
• Decision threshold — manually slide the classifier boundary left/right.
• 🎯 Optimize for Accuracy — automatically finds the threshold that maximizes accuracy. Watch it push toward predicting "healthy" for almost everyone as prevalence drops.
• 🏆 Optimize for F1 — finds the threshold that maximizes F1-score instead, trading some accuracy for far better recall of true disease cases.

Did you know?

A model that always predicts "no disease" for a condition with 2% prevalence is 98% accurate — and clinically worthless. That's the class-imbalance trap.