Linear Regression: Fitting a Plane to Data

Dataset

Noise level 0.8

Data points 80

Training

Learning rate 0.050

L2 regularization λ 0.00

Model State

ŷ = 0.00·x₁ + 0.00·x₂ + 0.00

Epoch

MSE loss

—

R² (fit quality)

—

How it works

Each point has two features (x₁, x₂) and a target y sampled from a hidden linear rule plus noise. The model predicts ŷ = w₁x₁ + w₂x₂ + b and is scored by mean squared error over all n points, optionally penalised for large weights (L2/ridge regularization):

ŷᵢ = w₁x₁ᵢ + w₂x₂ᵢ + b
L  = (1/n) Σ (ŷᵢ − yᵢ)² + λ(w₁² + w₂²)

Batch gradient descent moves every weight a small step opposite its gradient, each epoch, over the whole dataset at once:

∂L/∂w₁ = (2/n) Σ (ŷᵢ−yᵢ)x₁ᵢ + 2λw₁
∂L/∂w₂ = (2/n) Σ (ŷᵢ−yᵢ)x₂ᵢ + 2λw₂
∂L/∂b  = (2/n) Σ (ŷᵢ−yᵢ)

w ← w − η·∂L/∂w,   b ← b − η·∂L/∂b

Green plane — the model's current fitted plane; it starts near-flat at random weights and bends toward the data as training proceeds.
Grey wireframe — the true relationship the noisy data was generated from; a good fit makes the green plane hug it.
Point colour — residual size: green points sit close to the current plane, red points are poorly predicted.
Learning rate too high overshoots and can diverge; too low converges very slowly — watch the loss curve readout as you drag it.
L2 regularization shrinks the weights toward zero, trading a little training error for a flatter, more robust plane — useful when noise is high.

This is the exact algorithm behind ridge regression and the first gradient step every neural network layer performs — just with one linear unit instead of millions.

Fitted plane (training) True relationship High-residual point

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.