Presets
Playback
Cloud of points
Random points around the line y = 1 + 0.8 x.
Suspect point
Move one extra point anywhere.
Time
Speed of the bar settling.
Layers
Model checks
The science
The line as a mechanical equilibrium
Ordinary least squares chooses the line y = a + b x that minimises E = 1/2 sum (y_i - a - b x_i)^2. That is exactly the elastic energy of a rigid bar joined to every point by a vertical spring of unit stiffness. At equilibrium the net force sum e_i and the net torque sum x_i e_i on the bar vanish (the normal equations), where e_i = y_i - a - b x_i are the residuals. The page releases a damped bar from a flat position and lets it find that equilibrium.
Leverage
The fitted values are y_hat = H y with the hat matrix H = X (X'X)^-1 X'. Its diagonal, the leverage, is h_ii = 1/N + (x_i - x_mean)^2 / Sxx: it measures how strongly point i pulls its own fitted value. Leverages sum to the number of parameters (2) so the average is 2/N; a point much farther from the mean in x than the rest has leverage near 1 and the line is forced through it.
Residuals and influence
The standardised residual is r_i = e_i / sqrt(s^2 (1 - h_ii)) with s^2 = SSE / (N - 2). Cook's distance combines size of residual and leverage: D_i = e_i^2 h_ii / (2 s^2 (1 - h_ii)^2), which equals the change in all fitted values when point i is left out, divided by 2 s^2. A common rule of thumb flags D_i > 4/N. High leverage with a point on the trend gives small D; an outlier in the middle gives small D; a far point against the trend gives a large D.
Simplifications
One predictor, normal errors with equal variance, and a single suspect point; real data have many predictors (leverage then uses the full hat matrix) and rules of thumb for D are only guides. The bar dynamics are a teaching device with arbitrary mass and damping; only the equilibrium is the regression.
What to try
- With the default point at x = 7, y = 1, compare the slope with and without it: one point out of 21 has changed the slope noticeably. Its leverage is large and so is Cook's D.
- Press the far-point-on-the-trend preset: leverage stays high but the line barely moves and D is small.
- Press the outlier-in-the-middle preset: a huge residual with low leverage changes the intercept a little but not the slope.
- Drag the suspect y in graph 3's terms: the slope grows linearly with the suspect y, with a steeper rate when leverage is high.
- Use fewer cloud points (n = 8): the same suspect point gets more leverage and more influence.