Credit Scoring and Fraud Detection with Machine Learning: A Retail Banking Case Study

Two problems, one bank

A large retail bank with tens of millions of customers and a leading share of the domestic digital-banking market faced two distinct but related problems around 2019. On the lending side, 12–15% of loans were becoming non-performing (NPL), manual credit approval took one to three days — long enough that applicants routinely walked to a competitor — and the underlying scoring process relied heavily on loan officers' subjective judgement. On the fraud side, annual losses ran to the equivalent of $6–8 million, and fraud was almost always discovered after the money was already gone, because detection relied on periodic manual review rather than real-time monitoring.

These are two of the most common machine learning use cases in banking, and the bank's experience is a good template for any lender or payments business considering similar systems: the value comes from moving a decision from slow-and-subjective to fast-and-consistent, provided the organisation is willing to invest in the governance that comes with automating a regulated decision.

How the credit-scoring model worked

The scoring model ingested a loan application and combined several distinct feature groups: standard underwriting factors (income, credit history, age, occupation), digital footprint signals (frequency of app activity, spending category mix, transaction cadence in the bank's own mobile app), behavioural patterns at the point of application (time of day, how quickly the form was completed), and — more unusually — a social-graph signal, where a strong network of financially reliable existing customers around an applicant was treated as a modest positive factor.

The model output a score on a 0–1000 scale plus a recommended action: approve, decline, or refer for manual review. The headline operational change was decision speed: approvals that took one to three days dropped to roughly 90 seconds, because the model could return a confident, auditable recommendation instantly for the large majority of applications, reserving human review for genuinely ambiguous cases.

How the fraud-detection model worked

Every card transaction was scored in real time against a model of the customer's "normal" behaviour: typical location, time of day, merchant category, transaction size, and transaction frequency. When a transaction fell far enough outside that learned pattern, the system could block the card automatically and send an immediate confirmation prompt to the customer ("was this you?") rather than waiting for a periodic fraud review to catch it days or weeks later.

The technical stack combined scikit-learn and XGBoost for the underlying models, a streaming platform for real-time transaction scoring, and a feature store to keep the features used at inference time consistent with those used in training — a detail that matters enormously in fraud systems, where a mismatch between training-time and serving-time features silently degrades accuracy in a way that is hard to detect until losses climb.

Regulatory and ethical constraints that shaped the build

Two constraints fundamentally shaped the project, and both are worth planning for in advance rather than discovering midway through a build.

Explainability. The banking regulator required that a declined applicant be told, in substance, why they were declined — a plain "the model said no" is not acceptable in a regulated lending context. The bank's solution was to compute SHAP (SHapley Additive exPlanations) values for each decision, which attribute a prediction to individual input features and let the bank generate a human-readable summary of which factors most influenced a given score. Any team building a scoring or approval model in a regulated industry should budget explainability tooling as a core requirement from day one, not a bolt-on.

Fairness. A model trained on historical outcomes risks reproducing historical bias — for example, indirectly discriminating by age, gender or region through proxy variables even when those attributes are explicitly excluded. The bank instituted a recurring fairness audit and removed protected attributes and close proxies from the feature set, but even so had to rework parts of the model after an internal audit flagged disparate outcomes across demographic groups. That rework is a normal and expected part of deploying scoring models responsibly, not a sign the project failed.

Adapting the models to sudden economic disruption

A period of severe, rapid macroeconomic disruption — widespread job losses, large-scale internal migration, and a sharp change in typical spending patterns — tested both models at once. Historical creditworthiness signals became temporarily unreliable for a large segment of the customer base, and the bank had to retrain its credit model to account for what it internally called "disruption factors": a separate scoring track for internally displaced or relocated customers, elevated risk weighting for regions experiencing acute economic stress, and distinct criteria for hardship or bridge lending products.

Fraud patterns shifted at the same time, with a sharp rise in phishing and social-engineering attempts that exploited the general atmosphere of anxiety and disruption. The fraud model had to be retrained against these new attack patterns faster than its normal update cycle. The general lesson generalises well beyond banking: any business running a live scoring or anomaly-detection model should have a tested process for rapid retraining during a demand or risk-pattern shock, rather than assuming the model trained on "normal times" data will keep working through one.

Results and what they cost to achieve

Comparing 2019 (pre-ML) with the following one to two years of operation, the bank reported:

The build required a substantial team by the standards of most ML projects — twelve data scientists, eight ML engineers, four cybersecurity specialists and two product managers — with development cost in the low millions of dollars and ongoing monthly infrastructure and team cost in six figures. Reported payback was under three months, driven mainly by the combination of reduced loan losses and reduced fraud losses outweighing the build cost many times over in the first year, though a bank of this scale should be treated as an upper bound for what a smaller lender could expect — both the investment and the absolute savings scale with loan book size.

Lessons for other regulated businesses

Three lessons stand out for any business building automated decisioning in a regulated or high-stakes environment. First, pilot before you scale: the bank piloted the scoring model on roughly 5% of applicants, proved the hypothesis, and only then rolled it out fully — a much safer path than a full cutover. Second, explainability is not optional in regulated lending, and building it in from the start is cheaper than retrofitting it after a regulator or a wave of customer complaints forces the issue. Third, fairness auditing has to be continuous, not a one-time check before launch — the bank's own experience of having to rework a model after deployment shows that bias can surface even in a system that was checked carefully before going live.