SHAP Feature Attribution: Shapley Lattice & Convergence

Loan-Approval Instance

Income 0.40

Credit Score 0.60

Debt Ratio -0.30

Employment Length 0.20

Recent Inquiries -0.40

Interaction term (Income × Debt Ratio)

Monte-Carlo Sampling

Speed 6 steps/s

Live Readouts

Permutations sampled

Baseline E[f]

0.20

Prediction f(x)

0.00

Σ estimate → target

0.00

Feature	Exact φ	Estimate

How it works

SHAP explains a prediction by attributing it to features using the Shapley value from cooperative game theory. For a feature i and value function v(S) (the model's output using only the features in coalition S, others held at baseline):

φ_i = Σ_S⊆F\{i}  [|S|!(n-|S|-1)!/n!] · [v(S∪{i}) − v(S)]

Efficiency:  Σ φ_i = v(F) − v(∅) = f(x) − E[f(x)]

This 2D view draws the entire Boolean coalition lattice — all 32 subsets of the 5 features, arranged in columns by coalition size — and colors every node by its value v(S): blue above baseline, red below. A random permutation is sampled and animated as a bright path climbing left to right through the lattice, one feature at a time; each traversed edge is colored by the sign of the marginal jump it represents. Below the lattice, a convergence chart plots each feature's running-average estimated φ_i against the number of permutations sampled, with thin dashed lines marking the exact values (computed once by brute-force enumeration of all 32 subsets) that the Monte-Carlo estimate converges toward.

Lattice heat-map — every possible coalition's value, all at once; the empty set (∅) sits at far left, the full coalition (F) at far right.
Bright path — the single permutation currently being built; its edges are colored by whether adding that feature raised or lowered the coalition's value.
Convergence chart — five colored lines (one per feature) climbing toward their dashed exact-value targets as more permutations accumulate; by the additivity property they always sum to f(x) − E[f(x)].

This loan-approval model includes an interaction term between Income and Debt Ratio, so a feature's Shapley value is not just weight×value — it depends on which other features are already "in" the coalition, exactly like a real credit or clinical-risk model with correlated inputs.

Value above baseline / positive edge Value below baseline / negative edge Unvisited lattice node Current coalition (active path head)