Stacking Ensembles: Meta-Learner Blending

Stacked generalization combines several different models by training a second-level "meta-learner" on top of their predictions rather than simply averaging or voting. This simulator fits three genuinely different base learners — a kernel-weighted nearest-neighbor classifier, a gradient-descent logistic regression, and a threshold-search decision stump — on the same synthetic two-class dataset, then trains a logistic-regression meta-learner on their combined output as a real 3D bar-chart probability surface over the feature plane. Switching between naive in-sample stacking and honest K-fold cross-validated stacking reproduces one of the most common real-world ensembling mistakes: training the meta-learner on predictions that already leaked information about the training labels, which inflates fit accuracy while held-out test accuracy stays flat or drops — the generalization gap made visible.