Every orange (train) and blue (test) sphere sits at a noisy sample of a hidden function y(x₁,x₂). The translucent surface is a polynomial regression model, y = Σ wₖ·φₖ(x₁,x₂), whose weights are updated every frame by batch gradient descent on the training set's mean squared error — nudging the surface to pass closer to the orange points. A degree-1 model can only tilt a flat plane and underfits the true curved surface; a degree-3 model has enough terms (x₁³, x₁²x₂, …) to bend tightly around the training points, which lowers train MSE but can raise test MSE if it starts chasing noise instead of signal — the classic overfitting trade-off.
ŷ = Σₖ wₖ·φₖ(x₁,x₂) (φ = polynomial features)
MSE = (1/n)Σ(ŷᵢ − yᵢ)²
w ← w − η·∇MSE(w) R² = 1 − SSres/SStot
- Degree 1/2/3 — how many polynomial terms the model is allowed: linear-only, up to quadratic + interaction, or up to cubic. Higher degree fits training data more closely.
- Data noise — how much random scatter is added around the true function; more noise makes high-degree models more prone to overfitting it.
- Learning rate — the gradient-descent step size η; too low converges slowly, too high can make the fit oscillate instead of settling.
- Regenerate data — draw a fresh noisy train/test sample from the same hidden function and reset the model weights to watch it converge from scratch.
Real-world relevance: this train/test MSE gap is exactly what a data scientist watches for when choosing model complexity — the same principle behind cross-validation and regularization in real predictive-modeling pipelines for finance, healthcare and forecasting.