Permutation feature importance measures how much a trained model relies on one input feature, without touching the model itself. For feature j, on a held-out set of N samples:
PI_j = acc(baseline) − mean_k[ acc(X with column j shuffled) ]
for k = 1..K:
X'_j = random permutation of column j
keep all other columns and true labels fixed
acc_k = accuracy of model(X') vs true labels
PI_j = acc(baseline) − mean(acc_k), σ_j = std(acc_k)
Shuffling a feature destroys any real statistical relationship between that column and the label while leaving its marginal distribution and every other feature untouched. If the model actually depends on feature j, accuracy drops after the shuffle — the size of the drop is the importance score. If the model ignores feature j (pure noise), shuffling it changes almost nothing, so PI_j ≈ 0 (and can even be slightly negative from sampling noise).
- Feature buttons — choose which column's permutation to preview before running the full test.
- Repeats K — how many independent shuffles are averaged per feature; more repeats shrink the ± σ error bar.
- Dataset size / Label noise — regenerate the dataset with a different point count or amount of label noise; the baseline accuracy and importance ranking should stay qualitatively the same.
- Run Permutation Test — plays all 3×K shuffles in sequence: x₁/x₂ trials slide points across the scatter plot, x₃ trials slide points along the feature rail below it; correct points stay teal and newly-misclassified points turn red, and the bar for that feature grows toward its final mean importance.
- New Dataset — resamples points from the same generating rule so you can confirm the ranking is stable, not a fluke of one sample.
- Drag / scroll on the scatter plot — pan and zoom the view, exactly like rotating the camera around the old 3D point cloud, except here you are moving through the x₁–x₂ plane itself.
This dataset is generated so the true label mostly depends on x₁ and x₂ (weights 2.6 and 2.2) with x₃ contributing almost nothing (weight 0.15) — a built-in irrelevant feature. A correct permutation-importance run should reveal exactly that: large bars for x₁/x₂, a near-zero bar for x₃. This is the same technique scikit-learn's permutation_importance and R's vip package compute — model-agnostic, unlike SHAP or tree-specific gain scores.
Why x₃ lives on its own rail: the original 3D version spent one spatial axis per feature. A flat canvas only has two spatial axes, so x₁ and x₂ keep the scatter plot and x₃ gets its own 1D rail underneath — permuting it still visibly reshuffles every point's position on that rail and still recolors misclassifications exactly like the other two features.