Column stores (Parquet, ORC, and the engines behind Snowflake / Redshift / BigQuery) keep every row's value for one column contiguously, which makes two classic compression tricks cheap:
Dictionary encoding
build a table of the k distinct values,
replace each row with a ⌈log2 k⌉-bit index
size ≈ n·⌈log2 k / 8⌉ + k·avgBytes
Run-length encoding (RLE)
replace each maximal run of r equal
values with one (value, r) pair
expected runs R ≈ 1 + (n-1)(1-p)
size ≈ R·(avgBytes + runLenBytes)
Dictionary + RLE
RLE runs of dictionary indices instead
of raw values — compounds both savings
size ≈ R·(codeBytes + runLenBytes) + k·avgBytes
- Row count — how many values sit in the column (log scale, 1K–10M).
- Cardinality k — how many distinct values exist; a low-cardinality column (status flags, country codes) compresses hardest.
- Sort order p — the probability a value repeats its predecessor. Sorting or clustering a column before writing it is exactly what makes RLE effective: p → 1 means long runs, p → 0 means the value changes almost every row.
- The top strip is a live sample of the column: each block is one value, colour = which of the k distinct values it holds. RLE modes visually fuse a run into one wide block tagged ×N; dictionary modes shrink block height to show fewer bits per row and add a dictionary rack above. The bottom panel bar-charts encoded size across all four modes so you can compare them at a glance. Hover any strip block or bar for details.
Real-world relevance: this is why analytical warehouses ask you to sort or cluster tables by low-cardinality columns before loading — the same bytes on disk compress 10–100× better, which is most of why columnar formats outperform row stores for analytics.