Imbalanced Data: Why 99% Accuracy Can Mean 0% Recall

Imagine a fraud-detection model that scores 99.2% accuracy on its test set. Sounds excellent, until you learn that only 0.8% of transactions in the dataset are actually fraudulent. A model that simply predicts "not fraud" for every single transaction would score 99.2% too, while catching exactly zero fraud cases. This is the central trap of imbalanced data: when one class vastly outnumbers another, accuracy stops measuring what you think it measures.

Why accuracy breaks down

Accuracy is (correct predictions) / (total predictions). When 99% of your data belongs to one class, a model can reach 99% accuracy by learning almost nothing about the minority class, the one that usually matters most: fraud, disease, equipment failure, credit default. The metric rewards ignoring the signal you built the model to find.

Reading the confusion matrix

The fix starts with looking past a single accuracy number and into the four cells of a confusion matrix: true positives, false positives, true negatives, and false negatives. From these you get recall (of all actual positives, how many did we catch) and precision (of everything we flagged, how many were real). For a rare, high-stakes class, recall usually matters more than raw accuracy ever will.

Two common fixes

Class weighting tells the training algorithm to penalize mistakes on the minority class more heavily, so the model cannot get away with ignoring it. Oversampling duplicates (or synthesizes, as in SMOTE) minority-class examples until the training set is closer to balanced, forcing the model to actually learn their pattern rather than average them away.

Try it yourself

The Imbalanced Data Lab lets you skew a synthetic dataset toward one class, train a logistic-regression classifier with or without a fix applied, and watch the confusion matrix and minority-class recall change in real time.