HomeData ScienceColumnar Compression: Dictionary & Run-Length Encoding

Columnar Compression: Dictionary & Run-Length Encoding

Interactive 3D columnar-storage simulator: watch dictionary encoding and run-length encoding compress a data-warehouse column, with live byte counts and compression ratio as row count, cardinality and sort order change.

Data Science3DModerate60 FPS
ds-topic-100 ↗ Open standalone

Every analytical data warehouse — Snowflake, BigQuery, Redshift, or a raw Parquet file sitting in a data lake — leans on the same two tricks to shrink a column before it ever touches disk: dictionary encoding, which swaps repeated strings for small integer codes, and run-length encoding, which collapses a stretch of identical values into one (value, count) pair. This simulator renders a live sample of a data-warehouse column as a row of coloured 3D blocks and lets you tune row count, cardinality and how clustered/sorted the values are, watching the blocks fuse into runs or shrink into dictionary codes while the raw size, encoded size and compression ratio update from the same formulas real columnar engines use.

⚙ Under the hood

Interactive 3D columnar-storage simulator: watch dictionary encoding and run-length encoding compress a data-warehouse column, with live byte counts and compression ratio as row count, cardinality and sort order change.

big datadata warehousecompressioncolumnar storagedata engineeringparquet

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install

What did you find?

Add reproduction steps (optional)