Causal Inference for ML Practitioners

Causal inference answers "what if" questions: What would happen if we changed the price? Did the marketing campaign actually increase sales? Standard ML predicts correlations, but business decisions require understanding causes. Causal methods estimate treatment effects when randomized experiments are impossible or impractical.

Where It Is Used

Causal inference powers decision-making across industries:

• Marketing: Campaign attribution, channel effectiveness
• Product: Feature impact analysis, recommendation effects
• Healthcare: Treatment effectiveness, drug safety
• Policy: Program evaluation, intervention effects
• Finance: Credit decision impact, pricing strategy
• HR: Training program effectiveness, hiring policy effects

Use cases: Measuring true ad campaign ROI, understanding feature adoption impact, evaluating medical treatment effectiveness without RCTs.

Typical Data You Will See

Common data structure:

user_id | treatment | outcome | confounder_1 | confounder_2 | ... | timestamp

Key challenge: Confounding - variables that affect both treatment assignment and outcome, making naive comparisons biased.

Example: Users who see a premium feature are more engaged → but engaged users are more likely to see the feature. Correlation ≠ causation.

What You Do in Practice

1. Define the causal question - What treatment, what outcome, what comparison?
2. Draw causal diagram (DAG) - Map relationships between variables
3. Identify confounders - What affects both treatment and outcome?
4. Check identification - Can we estimate the causal effect?
5. Choose estimation method - Matching, weighting, IV, DiD based on data
6. Estimate propensity scores - If using weighting/matching
7. Balance checks - Verify covariate balance after adjustment
8. Estimate treatment effect - ATE, ATT, CATE as needed
9. Sensitivity analysis - How robust to unmeasured confounding?
10. Communicate uncertainty - Confidence intervals, assumptions made

Baseline First

Start with simple comparison, then add causal adjustments:

import pandas as pd
from sklearn.linear_model import LogisticRegression
from causalinference import CausalModel

# Step 1: Naive comparison (biased)
naive_effect = df[df['treatment']==1]['outcome'].mean() - \
               df[df['treatment']==0]['outcome'].mean()

# Step 2: Propensity score estimation
ps_model = LogisticRegression()
ps_model.fit(df[confounders], df['treatment'])
df['propensity'] = ps_model.predict_proba(df[confounders])[:, 1]

# Step 3: IPW estimation
df['weight'] = df['treatment'] / df['propensity'] + \
               (1 - df['treatment']) / (1 - df['propensity'])
ipw_effect = (df['weight'] * df['treatment'] * df['outcome']).sum() / \
             (df['weight'] * df['treatment']).sum() - \
             (df['weight'] * (1-df['treatment']) * df['outcome']).sum() / \
             (df['weight'] * (1-df['treatment'])).sum()

Why propensity score baseline: - Intuitive: balance treatment groups - Flexible: works with any outcome model - Diagnostic: balance checks validate approach - Foundation for more complex methods

Causal Estimation Methods

Decision guide: - Have all confounders measured? → PSM, IPW, or Doubly Robust - Policy change at specific time? → Difference-in-Differences - Assignment based on threshold? → Regression Discontinuity - Unmeasured confounding suspected? → Instrumental Variables - One treated unit (e.g., country)? → Synthetic Control

Key Concepts

Average Treatment Effect (ATE):

ATE = E[Y(1)] - E[Y(0)]

Effect for entire population.

Average Treatment Effect on Treated (ATT):

ATT = E[Y(1) - Y(0) | T=1]

Effect for those who actually received treatment.

Conditional Average Treatment Effect (CATE):

CATE(x) = E[Y(1) - Y(0) | X=x]

Heterogeneous effects by subgroup.

Propensity Score:

e(x) = P(T=1 | X=x)

Probability of treatment given covariates.

Ignorability/Unconfoundedness:

Y(0), Y(1) ⊥ T | X

Treatment assignment is as-if-random conditional on X.

Evaluation and Validation

Causal methods can't be validated on observational data alone (no ground truth counterfactual). Use:

Balance check example:

# Standardized mean difference before/after weighting
smd_before = (treated_mean - control_mean) / pooled_std
smd_after = (weighted_treated_mean - weighted_control_mean) / pooled_std
# Target: |SMD| < 0.1 after adjustment

Failure Modes and Debugging Checklist

• [ ] Unmeasured confounding: Key confounder not in data
• [ ] Positivity violation: No overlap in propensity scores
• [ ] Model misspecification: Propensity/outcome model wrong
• [ ] Selection bias: Sample not representative
• [ ] Reverse causality: Outcome affects treatment
• [ ] Time confounding: Treatment timing correlates with trends
• [ ] Interference: Treatment of one unit affects another
• [ ] Survivorship bias: Only observing units that "survived"

Production Checklist

Before making decisions: - [ ] Causal diagram reviewed by domain experts - [ ] Confounders identified and measured - [ ] Propensity overlap verified - [ ] Balance achieved after adjustment - [ ] Multiple methods give consistent results - [ ] Sensitivity analysis conducted - [ ] Assumptions documented

When deploying causal ML: - [ ] Treatment effect estimates monitored - [ ] Covariate distributions tracked - [ ] Periodic validation against experiments - [ ] Decision rules based on effect magnitude + uncertainty - [ ] Audit trail for causal claims

Organizational: - [ ] Stakeholders understand assumptions - [ ] Limitations clearly communicated - [ ] Preference for experiments when feasible - [ ] Causal claims reviewed before publication

Tools and Stack

Example Mini-Case

Scenario: E-commerce platform wants to measure impact of showing personalized recommendations on purchase rate, but can't run A/B test (recommendations already deployed).

Data: 500K users, recommendation exposure logged, purchase outcomes, user features (history, demographics).

Approach: 1. Draw DAG: User history → recommendation exposure AND purchase 2. Identify confounders: Purchase history, browse frequency, account age 3. Estimate propensity scores: LogisticRegression on confounders 4. Check overlap: Trim extreme propensities (< 0.05 or > 0.95) 5. IPW estimation: Weighted ATE 6. Doubly robust: AIPW for robustness 7. Sensitivity: E-value for unmeasured confounding

Results: - Naive difference: +8% purchase rate - IPW estimate: +3.2% (95% CI: 2.1% - 4.3%) - Doubly robust: +3.0% (95% CI: 1.9% - 4.1%) - E-value: 1.8 (moderate robustness to unmeasured confounding)

Conclusion: Recommendations causally increase purchases by ~3%, not 8% as naive comparison suggested. Confounding by user engagement explained the difference.

Key learnings: - Naive estimates significantly biased - Multiple methods increased confidence - Sensitivity analysis essential for communication - Later validated with holdout A/B: 2.8% lift (consistent)

Related Pages

• Index: ML Field Guide
• A/B Testing for ML
• Uplift Modeling
• Feature Engineering Fundamentals

Further Reading

• DoWhy Documentation
• EconML User Guide
• Causal Inference: The Mixtape
• The Book of Why - Judea Pearl
• Mostly Harmless Econometrics

Try it live

Everything above runs in your browser — open Causal Inference Lab — Confounding, Naive vs. Adjusted Effects and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.