SHAP Feature Attribution: Shapley Lattice & Convergence

What SHAP Feature Attribution Is

SHAP (SHapley Additive exPlanations) is an interpretable machine learning technique that provides consistent, game-theoretic attribution of feature importance to individual predictions. It uses the Shapley value from cooperative game theory to explain the output of any model in terms of its input features.

The SHAP method calculates a unique contribution for each feature by considering all possible combinations (coalitions) of features and averaging their marginal contributions, ensuring that the total attribution sums up to the prediction difference between the actual instance and the baseline.

How the Shapley Lattice Works

The Shapley lattice is a structured representation of all possible feature combinations in a model. In a binary feature space, it forms a Boolean coalition lattice where each node represents a unique set of active features. The SHAP values are computed for each node and then mapped onto this lattice to visualize the contribution of each feature.

By heat-mapping these values on the Shapley lattice, one can easily see which combinations of features have the most significant impact on model predictions.

Monte-Carlo Permutation Animation

The Monte-Carlo permutation animation in SHAP feature attribution is a dynamic visualization that helps understand how each feature contributes to the prediction. It works by randomly shuffling (permuting) features and observing the change in model output, which provides an estimate of the feature's importance.

This process is animated on the Shapley lattice, showing a bright path climbing one feature at a time, illustrating the convergence of SHAP values to their exact game-theoretic attributions.

Convergence Chart

The convergence chart in SHAP feature attribution displays the running estimate of each feature's importance as the Monte-Carlo permutations progress. As more permutations are performed, the estimates settle onto their exact game-theoretic attributions, providing a clear visual indication of how accurately the SHAP values are calculated.

This chart is particularly useful for understanding the stability and reliability of the SHAP values over multiple iterations.

Frequently asked questions

Try it live

Everything above runs in your browser — open SHAP Feature Attribution: Shapley Lattice & Convergence and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.