Home▸AI & Machine Learning▸Data Science Big Data Simulation

🧪 Data Science Big Data Simulation

Interactive 3D simulator of external k-way merge sort — the technique big-data systems use to sort datasets too large for memory. Split a dataset into sorted runs, merge them with a min-comparison sweep, and watch live comparison counts and I/O-pass stats.

AI & Machine Learning2DModerate60 FPS📱 Mobile-adapted

Real big-data pipelines routinely sort datasets far larger than available memory — Hadoop's shuffle, Spark's sort-based shuffle, and every database's external sort all use the same trick: split the data into chunks that fit in memory, sort each chunk into a "run," then merge the sorted runs in a single sequential pass. This simulator makes that two-phase process visible in 3D. Each colored column is one sorted run of the dataset; on every merge step, the algorithm compares the smallest not-yet-emitted record at the head of each run, picks the overall smallest, and appends it to the growing merged output column, tracking live comparison counts, merge progress, and the constant single-pass I/O cost that makes external merging so much cheaper than re-sorting everything from scratch.

⚙ Under the hood

This simulation introduces the core concepts of big data science, including data collection, processing, and analysis using simulated datasets. It focuses on foundational techniques.

Data ScienceBig Data

2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install