Churn Prediction

Churn prediction identifies customers likely to leave before they do. It is one of the highest-ROI ML applications - retaining existing customers costs 5-25x less than acquiring new ones. Good churn models enable proactive retention campaigns targeted at the right customers at the right time.

Where It Is Used

Churn prediction is critical across subscription and service businesses:

• Telecom: Mobile and internet service cancellation
• SaaS: Software subscription cancellation
• Streaming: Netflix, Spotify subscription churn
• Banking: Account closure, credit card cancellation
• Gaming: Player drop-off prediction
• Retail: Customer lapse prediction

Companies: Major telecom providers, all subscription businesses, loyalty programs.

Typical Data You Will See

Data sources: - Usage data (logins, transactions, engagement) - Billing and payment history - Customer service interactions - Account information and tenure - Marketing touchpoints - External data (demographics, market)

What You Do in Practice

1. Define churn - What exactly constitutes churn for your business?
2. Set prediction horizon - How far ahead to predict?
3. Create observation windows - Historical features and label periods
4. Engineer features - Usage trends, engagement, behavioral signals
5. Handle imbalance - Churn is typically minority class
6. Train model - Classification or survival model
7. Calibrate probabilities - Accurate scores for business decisions
8. Evaluate business impact - Revenue saved, not just AUC
9. Deploy retention actions - Trigger campaigns based on risk
10. Measure intervention effectiveness - Did actions reduce churn?

Baseline First

Start with simple rules before ML:

# Rule-based baseline
def simple_churn_rules(customer):
    """Basic churn indicators."""
    risk_score = 0

    # No activity in last 30 days
    if customer.days_since_last_activity > 30:
        risk_score += 3

    # Usage declining
    if customer.usage_trend_30d < -0.3:
        risk_score += 2

    # Support complaints
    if customer.complaints_last_90d > 2:
        risk_score += 2

    # Payment issues
    if customer.failed_payments_last_90d > 0:
        risk_score += 2

    return risk_score >= 4  # High risk threshold

Why rule baseline: - Domain experts know key churn signals - Interpretable and easy to validate - Establishes what ML needs to beat - May be sufficient for small scale

Feature Engineering

Churn features typically come from behavioral patterns:

Temporal feature engineering:

def create_churn_features(customer_id, observation_date):
    """Create features for churn prediction."""
    features = {}

    # Activity windows
    for window in [7, 14, 30, 60, 90]:
        activity = get_activity(customer_id, 
                               end=observation_date,
                               days=window)
        features[f'logins_{window}d'] = activity.login_count
        features[f'sessions_{window}d'] = activity.session_count
        features[f'duration_{window}d'] = activity.total_duration

    # Trends (compare recent to historical)
    recent = get_activity(customer_id, end=observation_date, days=14)
    historical = get_activity(customer_id, 
                              end=observation_date - timedelta(14),
                              days=14)

    if historical.login_count > 0:
        features['login_trend'] = (
            recent.login_count - historical.login_count
        ) / historical.login_count
    else:
        features['login_trend'] = 0

    # Recency
    last_activity = get_last_activity(customer_id, observation_date)
    features['days_since_activity'] = (
        observation_date - last_activity
    ).days

    # Billing
    features['payment_failures_90d'] = count_payment_failures(
        customer_id, observation_date, days=90
    )

    return features

Handling Class Imbalance

Churn is typically 2-10% of customers - severe imbalance:

from imblearn.over_sampling import SMOTE
from imblearn.under_sampling import RandomUnderSampler
from imblearn.pipeline import Pipeline

# Option 1: SMOTE oversampling
smote = SMOTE(sampling_strategy=0.3)
X_resampled, y_resampled = smote.fit_resample(X_train, y_train)

# Option 2: Class weights
model = XGBClassifier(
    scale_pos_weight=len(y_train[y_train==0]) / len(y_train[y_train==1])
)

# Option 3: Combined approach
pipeline = Pipeline([
    ('under', RandomUnderSampler(sampling_strategy=0.5)),
    ('over', SMOTE(sampling_strategy=0.8))
])

# Option 4: Cost-sensitive threshold
# Don't resample, adjust decision threshold based on costs

Threshold optimization:

def optimize_threshold(y_true, y_proba, 
                       cost_fn=1, cost_fp=0.1, cost_tp=-0.5):
    """Find threshold that minimizes business cost."""
    thresholds = np.arange(0.1, 0.9, 0.01)
    costs = []

    for thresh in thresholds:
        y_pred = (y_proba >= thresh).astype(int)

        fn = ((y_true == 1) & (y_pred == 0)).sum()  # Missed churners
        fp = ((y_true == 0) & (y_pred == 1)).sum()  # False alarms
        tp = ((y_true == 1) & (y_pred == 1)).sum()  # Caught churners

        total_cost = fn * cost_fn + fp * cost_fp + tp * cost_tp
        costs.append(total_cost)

    best_idx = np.argmin(costs)
    return thresholds[best_idx]

Model Approaches

Gradient Boosting example:

from lightgbm import LGBMClassifier

model = LGBMClassifier(
    n_estimators=200,
    learning_rate=0.05,
    num_leaves=31,
    class_weight='balanced',
    random_state=42
)

model.fit(X_train, y_train)

# Probability predictions for risk scoring
churn_proba = model.predict_proba(X_test)[:, 1]

Survival Analysis approach:

from lifelines import CoxPHFitter

# Prepare data with duration and event
df['duration'] = (df['observation_date'] - df['signup_date']).dt.days
df['churned'] = df['churn_date'].notna().astype(int)

# Cox Proportional Hazards
cph = CoxPHFitter()
cph.fit(df[features + ['duration', 'churned']], 
        duration_col='duration', 
        event_col='churned')

# Survival probability at 30 days
survival_30d = cph.predict_survival_function(
    df[features], times=[30]
).T
churn_prob_30d = 1 - survival_30d

Evaluation and Metrics

Business-focused evaluation:

def calculate_lift_chart(y_true, y_proba, n_bins=10):
    """Calculate lift by decile."""
    df = pd.DataFrame({'true': y_true, 'proba': y_proba})
    df['decile'] = pd.qcut(df['proba'], n_bins, labels=False)

    lift = df.groupby('decile').agg({
        'true': ['sum', 'count', 'mean']
    })
    lift.columns = ['churners', 'total', 'churn_rate']
    lift['lift'] = lift['churn_rate'] / y_true.mean()

    return lift

# Top decile should have highest concentration of churners
lift_chart = calculate_lift_chart(y_test, churn_proba)
print(f"Lift at top decile: {lift_chart.loc[9, 'lift']:.2f}x")

Failure Modes and Debugging Checklist

• [ ] Leakage: Using churn-related features (cancellation calls)
• [ ] Wrong observation window: Features computed after prediction point
• [ ] Survivorship bias: Only analyzing current customers
• [ ] Definition drift: Churn definition changes over time
• [ ] Intervention leakage: Past retention actions in features
• [ ] Seasonality: Not accounting for seasonal patterns
• [ ] Segment blindness: Model works for average, fails for segments

Production Checklist

Before deployment: - [ ] Churn definition validated with business - [ ] Feature pipeline handles missing data gracefully - [ ] Model calibrated for probability accuracy - [ ] Threshold optimized for business costs - [ ] Intervention strategy defined per risk tier

After deployment: - [ ] Predictions generated on schedule (daily/weekly) - [ ] Risk scores pushed to CRM/marketing systems - [ ] Model performance monitored vs actuals - [ ] Intervention effectiveness tracked - [ ] Feature drift monitored

Intervention integration: - [ ] Risk tiers defined (high/medium/low) - [ ] Retention actions per tier (discount, outreach, content) - [ ] Holdout group for measuring intervention impact - [ ] Feedback loop to model (what worked?)

Measuring Intervention Impact

Running retention campaigns requires measuring true impact:

def measure_intervention_impact(treatment_group, control_group):
    """Compare churn rates between groups."""
    treatment_churn = treatment_group['churned'].mean()
    control_churn = control_group['churned'].mean()

    lift = (control_churn - treatment_churn) / control_churn

    # Statistical significance
    from scipy.stats import chi2_contingency
    contingency = pd.crosstab(
        pd.concat([treatment_group, control_group])['group'],
        pd.concat([treatment_group, control_group])['churned']
    )
    chi2, p_value, _, _ = chi2_contingency(contingency)

    return {
        'treatment_churn': treatment_churn,
        'control_churn': control_churn,
        'relative_reduction': lift,
        'p_value': p_value
    }

Tools and Stack

Example Mini-Case

Scenario: SaaS company with 50K customers, 8% monthly churn, $100 average monthly revenue.

Approach: 1. Definition: No login in 30 days after billing date 2. Features: 120 features from usage, billing, support 3. Model: LightGBM with class weights 4. Threshold: Optimized for 3:1 FN:FP cost ratio

Results: - AUC: 0.82 - Lift at top decile: 4.2x - Precision at top 10%: 31% (vs 8% baseline) - Intervention: 15% discount for high-risk customers

Business impact: - 200 high-risk customers identified monthly - 40% discount acceptance rate - 25% churn reduction among intervened - Net revenue saved: $12K/month (after discount cost)

Related Pages

• Index: ML Field Guide
• Binary Classification
• Customer Segmentation
• A/B Testing for ML

Further Reading

• Fighting Churn with Data
• Survival Analysis for Customer Analytics
• Cost-Sensitive Learning
• Uplift Modeling for Targeting

Try it live

Everything above runs in your browser — open Customer Churn Predictor — Survival Curve Live and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.