Home▸Data Science▸The Hadoop Ecosystem: HDFS, MapReduce & YARN

🐘 The Hadoop Ecosystem: HDFS, MapReduce & YARN

A 3D data-centre model of a Hadoop cluster: watch HDFS split and replicate a file into blocks across DataNodes, then run a MapReduce job and see YARN schedule map, shuffle and reduce tasks onto the nodes that already hold the data.

Data Science3DAdvanced60 FPS

A 3D data-centre model of a Hadoop cluster: HDFS splits an input file into blocks and replicates them across DataNodes, then a MapReduce job runs with YARN scheduling map tasks onto the nodes that already hold the data before shuffling and reducing the results.

🔬 What It Demonstrates

Why Hadoop separated storage (HDFS), scheduling (YARN) and computation (MapReduce) into three cooperating layers, and how "moving computation to the data" avoids shipping huge files across the network.

🎮 How to Use

Adjust cluster size, replication factor and file blocks to reshape HDFS storage, then press Run MapReduce job and watch map, shuffle and reduce phases sweep across the cluster. Toggle the view to see YARN's container allocation instead of raw blocks.

💡 Did You Know?

Before YARN (Hadoop 2.0, 2013), MapReduce's JobTracker handled both resource scheduling and job logic — a bottleneck YARN removed by becoming a general-purpose cluster scheduler that today also runs Spark, Tez and other frameworks.

⚙ Under the hood

hadoophdfsmapreduceyarnclusterdistributed computingThree.js

3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install