The Call-Duration Trap: How Data Leakage Skews Bank Marketing Prediction Models

A deposit-prediction problem with a hidden shortcut

A well-known dataset from a Portuguese retail bank records the outcome of roughly 41,000 outbound telemarketing calls, each one an attempt to sell a term deposit. The task looks like a textbook binary classification problem: given everything the bank knows about a customer and the campaign, predict whether the column y will end up 'yes' or 'no'. Roughly 21 fields are available, split across three natural groups — client attributes such as age, job, marital status and existing loans; campaign-contact details such as the communication channel, the month, and how many times the person has been called before; and a handful of macroeconomic indicators describing the state of the economy at the time of the call.

Buried in the campaign-contact group is a field called duration: the length, in seconds, of the last phone call. It turns out to be by far the strongest single predictor of a 'yes' outcome. Long calls correlate enormously with sales, because a call that ends in seconds is almost always a hang-up, while a call that runs for several minutes usually means the customer stayed on the line to ask questions about the product. Any model given this feature will happily learn that rule and post excellent accuracy and ROC-AUC scores on held-out data.

Why the strongest predictor still has to go

The catch is a matter of timing rather than statistics. duration is only known once the call has already finished. A model meant to help a call-centre manager decide who to phone, or in what order, needs its inputs to be knowable before the call happens. Feeding it the call's own length is a form of data leakage: information about the future outcome sneaking into the input via a feature that will not actually exist at prediction time in production.

This is a specific and common flavour of leakage — sometimes called a 'post-outcome' or 'downstream' feature. It differs from more obvious leaks (like accidentally including the label itself, or a proxy for it computed after the fact) mainly in how innocuous it looks. duration is a perfectly ordinary integer column sitting next to campaign and previous, and nothing in the schema flags it as dangerous. The only way to catch it is to reason carefully about the causal and temporal order of events: does the value exist at the moment the prediction actually needs to be made? For duration, the answer is clearly no, so responsible practice on this dataset is to drop it entirely before training the model that will be deployed, even though doing so measurably lowers every headline metric.

What actually predicts a 'yes' once the shortcut is removed

With duration excluded, feature-importance rankings and SHAP (SHapley Additive exPlanations) analysis on a gradient-boosted model point to a different, more useful set of drivers. Macroeconomic indicators dominate: the three-month Euribor interest rate, the number of people employed nationally, and the employment variation rate together carry the most explanatory weight. That makes intuitive sense — a term deposit is essentially competing for the customer's money against other uses of that cash, and when interest rates and the broader economy shift, the relative appeal of locking money away in a deposit shifts with them. SHAP summary plots on this dataset typically show that higher Euribor values push predictions down, meaning that when short-term borrowing/lending rates are elevated elsewhere, customers are somewhat less inclined to commit to this particular bank's deposit offer relative to other windows in the campaign.

Below the macro variables, the outcome of any previous marketing contact with the same customer (poutcome) is highly informative — someone who said yes to a past campaign is much likelier to say yes again. The number of contacts made during the current and prior campaigns, the customer's age (with a non-linear relationship, since very young and retired customers behave differently from prime-working-age customers), and education level round out the top predictors.

Reframing the task as ranking, not classification

Because the true production feature set excludes anything generated during or after the call, the practical version of this task is closer to a ranking problem than a strict classification problem: given only pre-call information, produce a probability score for every customer in the calling list, then contact people in descending order of predicted likelihood. This reframing matters for model selection, too. A model that is only 'reasonably' accurate at binary yes/no classification can still be extremely useful if it is well-calibrated enough to rank customers sensibly, since a call centre with limited capacity mainly needs to know who to prioritise, not an exact hit-or-miss verdict for each name on the list.

Why plain accuracy is the wrong yardstick here

Only around 12% of contacted customers actually subscribe to a term deposit; the remaining 88% decline. A model that predicts 'no' for every single customer would already be roughly 88% accurate while being completely useless for the bank's actual goal of finding the minority who will say yes. This is the class-imbalance problem that shows up constantly in marketing-response, fraud and churn modelling, and it is why practitioners lean on metrics that are far less forgiving of majority-class laziness: ROC-AUC, which measures how well the model ranks positive cases above negative ones across every possible decision threshold; precision and recall, which separately capture how many predicted 'yes' customers were correct versus how many actual 'yes' customers were found; the F1-score, their harmonic mean; and precision-recall AUC, which is generally considered more informative than ROC-AUC specifically when positives are rare, because it does not get inflated by the sheer volume of easy true negatives.

Lessons that generalise beyond one dataset

The duration trap is a useful case study precisely because it is easy to miss and easy to explain once found. The general habit it teaches is to ask, for every candidate feature, exactly when in the real-world process that value becomes available, and to compare that moment to when the prediction actually needs to be made. Features generated as a side-effect of the outcome itself — a call's length, a support ticket's resolution time, a transaction's final status — are common offenders in fields well beyond banking, including healthcare readmission models and e-commerce conversion models. Catching them before deployment is what separates a model that performs brilliantly in a notebook from one that actually helps a call-centre manager decide who to phone next.