Each gold bar is the single number a chosen encoding hands to the model for that category; the small dots are the real per-row target values the bar is trying to summarise. Switch encodings to see the trade-off change:
One-Hot: 1 column per category ā sparse, no false order
Label: index 0..N-1 ā implies order that isn't real
Frequency: count(cat)/N ā leaks popularity, not target
Target: mean(target | cat) ā powerful, but noisy on small cats
- Cardinality ā how many distinct categories exist. One-hot's column count scales with this; other encodings stay at 1 column regardless.
- Samples per category ā target encoding's bar is a sample mean, so its standard error shrinks as 1/ān. Drag it low and watch the translucent uncertainty band around each target bar blow up ā that's the overfitting risk of target encoding on rare categories.
- Implied order corr. ā label encoding assigns 0,1,2ā¦ in category order, so it always correlates perfectly with that arbitrary index. A linear model reading that number will treat categories as ordered even when they aren't.
Real-world relevance: choosing an encoding is a genuine feature-engineering decision ā one-hot is safe but expensive at high cardinality, label encoding silently invents order, frequency encoding conflates popularity with signal, and target encoding is the strongest predictor but needs cross-validation or smoothing to avoid leaking the label into the feature.