Loading…
🌲 Random forest
Forest
Query point
View
The science

Bagging. Each tree is grown on a bootstrap sample: n draws with replacement. A sample is left out of a given draw with probability (1 − 1/n)n → 1/e ≈ 36.8 %, so every tree has about a third of the data it never saw (its out-of-bag, OOB, samples: pale small spheres when you inspect a tree).

Trees. CART classification trees. A split minimises the weighted Gini impurity G = 1 − Σpk² over all thresholds of mtry randomly chosen features, so trees differ. A fully grown tree memorises its bootstrap sample.

Forest and OOB error. The forest predicts by majority vote. Each sample is voted on only by the trees that did not see it; the share of wrong votes is the OOB error, a built-in estimate of test error. The page also scores 1500 fresh points from the same distribution (dashed line) for comparison.

Importance. Permutation importance on OOB samples: shuffle one feature among a tree's OOB samples and measure the drop in its accuracy. Here x and y separate the classes, z barely does, and the numbers show it.

Simplifications. Synthetic Gaussian data in a unit cube; hard-vote majority; no class weights; a single run per setting, so small differences between slider positions are within sampling noise. Only the first 36 trees are drawn as 3D trees (tree height = its depth).

Running numerical checks…

Forest

OOB error (forest)–
Test error (1500)–
Inspected tree error–
Tree size–
Tree saw–
OOB-scored–
Votes A / B / C–
Prediction–
Importance x / y / z–
Drag to orbit. Tree foliage shows each tree's vote for the white query sphere.
Made with 🟠 Blender · Three.js