Decision boundary — train the forest to see it
Individual trees (first 5) vs ensemble
Tree 1
Tree 2
Tree 3
Tree 4
Tree 5
This simulation trains a real random forest classifier in your browser: an ensemble of up to 100 decision trees, each grown on its own bootstrap sample (sampling with replacement from the dataset) and splitting on a random subset of features — by default √p of them, following Breiman's original algorithm. Each tree greedily picks the split that maximises impurity reduction, measured by Gini impurity or entropy, down to a maximum depth. Points left out of a tree's bootstrap sample become that tree's out-of-bag (OOB) set, letting the forest estimate its own generalisation error without a separate test set — a genuine implementation, not a canned animation.
A 2D binary/multi-class classification problem (Moons, Circles, Blobs or XOR) with a live decision boundary shaded by the forest's averaged class probabilities. Five of the individual trees are rendered as small thumbnails alongside the ensemble so you can see how each single tree overfits its own bootstrap sample while the aggregated vote smooths those errors into one boundary. A feature-importance bar for each of the two input dimensions is computed from the total impurity reduction each feature contributes across every split in every tree.
Choose a dataset shape and noise level, then set Trees (1–100), Max depth (1–12), Min samples split, Features per split (1, √p, or all) and the splitting Criterion (Gini or entropy). Press Train Forest to bootstrap-sample the data and grow every tree; the progress bar and per-tree thumbnails update as training runs. Try the Single tree preset to see one deep, overfit tree, then compare it with Small forest or Large forest to watch the boundary stabilise as more trees vote. Left-click the canvas to add a class-0 point, right-click for class 1, and retrain to include it.
Leo Breiman formalised the random forest algorithm in 2001 by combining bagging (bootstrap aggregating) with random feature selection at each split — two sources of randomness that decorrelate the trees so their errors partially cancel when averaged. Because roughly 37% of the data (1/e in the large-sample limit) is excluded from any given bootstrap sample, those out-of-bag points give the forest a built-in, unbiased error estimate, which is exactly what the OOB error stat in this simulation reports.
A random forest is an ensemble of many decision trees whose predictions are combined by averaging (for probabilities) or majority vote (for classes). A single decision tree, especially a deep one, tends to overfit its training data and is sensitive to small changes in it. By training each tree on a different bootstrap sample and letting it consider only a random subset of features at each split, the forest builds trees that make different mistakes; averaging those predictions cancels out much of the individual overfitting, producing a smoother, more reliable decision boundary than any single tree in the ensemble.
Bootstrap aggregating means drawing a new training set for each tree by sampling the original dataset with replacement, so the bootstrap sample is the same size as the original but contains duplicates and omissions. In this simulation, every tree gets its own bootstrap sample generated from a seeded random number generator before it is grown, exactly as in the original bagging procedure introduced by Leo Breiman in 1996 and later extended into random forests in 2001.
At every node, the tree first restricts itself to a random subset of the two available features according to the Features per split setting (1, √p ≈ 1 for a 2-feature problem, or all). For each candidate feature it tests every midpoint between adjacent sorted values as a threshold and computes the impurity reduction — the parent's Gini impurity or entropy minus the weighted impurity of the two child subsets. The split with the largest impurity reduction is chosen, and the process repeats recursively until a node is pure, reaches Max depth, or falls below Min samples split.
Both are measures of how mixed the classes are in a node: Gini impurity is 1 minus the sum of squared class proportions, while entropy is the sum of −p·log₂(p) over classes. Both are 0 for a pure node and reach their maximum when classes are evenly mixed. Selecting either as the Criterion in this simulation changes which candidate split is scored best at each node, but in practice the two criteria usually produce very similar trees — Gini is marginally faster to compute and entropy is slightly more sensitive to class balance.
Because each tree's bootstrap sample omits roughly a third of the data points, those omitted points act as a free validation set for that tree. The simulation collects, for every data point, the votes only from the trees that did not see it during training, and compares the majority vote against the true label to compute the OOB error shown in the Stats panel. This gives an honest estimate of how the forest would perform on unseen data without needing to hold out a separate test set — a key practical advantage of the random forest algorithm.