SHAP Values
LIME
Attention Weights
SHAP Feature Importance
LIME Local Explanation
Attention Heatmap
Sample Selection
Explanation Type
Export
Model Interpretability
Understanding WHY a model makes predictions is crucial for trust, debugging, and compliance. Interpretability tools explain black-box models.
SHAP (SHapley Additive exPlanations)
- Based on game theory (Shapley values)
- Shows contribution of each feature
- Consistent, theoretically grounded
- Works with any model
- Can be computationally expensive
LIME (Local Interpretable Model-agnostic Explanations)
- Explains individual predictions
- Fits simple model locally
- Model-agnostic
- Fast and intuitive
Other Methods
- Saliency Maps: Which pixels matter for CNNs
- GradCAM: Class activation mapping
- Attention Weights: For transformers
- Counterfactuals: "If X changed, prediction would flip"
Why Interpretability Matters
- Trust: Users trust explainable systems
- Debugging: Find why model fails
- Compliance: GDPR "right to explanation"
- Fairness: Detect bias in decisions
- Science: Discover insights from learned patterns