Home▸AI & Machine Learning▸Inside Apache Spark's Distributed Execution Model

⚡ Inside Apache Spark's Distributed Execution Model

A 3D driver/executor cluster that stages, shuffles and reduces data partitions on demand, showing exactly what happens when a lazy Spark DAG finally executes.

AI & Machine Learning3DAdvanced60 FPS

A driver coordinates a ring of executors as data partitions flow through a map stage, a network shuffle and a reduce stage — nothing happens until you fire the action that triggers Spark's lazily built DAG.

🔬 What It Demonstrates

Narrow transformations run in place on each executor with zero network cost; wide transformations force a shuffle, redistributing partitions by hash key before a reduce stage aggregates and the driver collects the result.

🎮 How to Use

Set the executor and partition counts, then press Run job to trigger the lazy DAG. Toggle in-memory caching to see mapped partitions stay resident on their executor for the next run instead of recomputing.

💡 Did You Know?

The shuffle is Spark's most expensive operation because it moves data across the network between every executor pair — minimizing shuffles is one of the biggest levers for speeding up a real Spark job.

⚙ Under the hood

A 3D driver/executor cluster that stages, shuffles and reduces data partitions on demand, showing exactly what happens when a lazy Spark DAG finally executes.

machine learningdata processingdistributed systemssparkdagexecution modelThree.js

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install