The classic UCI/Kaggle "Bank Marketing" dataset logs the outcome of a telemarketing campaign for a term-deposit product, together with the duration of each phone call in seconds. Duration turns out to be by far the single strongest predictor of a "yes" — long, engaged conversations convert; calls that are hung up in a few seconds don't. The problem: nobody knows how long a call will last before they dial the number. Using it as a training feature lets the model peek at information that is only produced by the outcome itself.
Dropping duration from this dataset typically drops a gradient-boosted
model's held-out AUC from roughly 0.93–0.95 down to around 0.70–0.75 — a huge fall in
reported accuracy, but the honest number is the only one that describes performance
you could actually get in production, before any call has been made.
A SHAP-style feature-importance chart and a live call timeline show what happens when a model is allowed to see how long a phone call lasted — information that, by definition, doesn't exist until after the call is already over.
Toggling call duration into the feature set makes it dominate every other predictor and drives the model's reported AUC from a modest ~0.71 up to a suspicious ~0.93 — a textbook sign of data leakage rather than genuine predictive skill.
Scrub or auto-play the call duration slider and watch the probability board react. With the leaky model on, longer calls send predicted probability soaring; switch it off and the same slider does nothing, because an honest, pre-call model was never shown the duration at all.
The UCI Bank Marketing dataset's own documentation explicitly warns that duration should be dropped if the goal is a realistic predictive model, since it "is not known before a call is performed."