One-Way ANOVA

Between-group against within-group variance, F and p as samples arrive.

Switch to 3D

Presets

Playback

Design

How many groups and how many samples in each.

Effect and noise

True group means are evenly spaced; sigma is the spread inside each group.

Time

How fast the sampling rounds arrive.

Layers

Model checks

The science

Splitting the variance

With k groups of m samples (N = k m) the total variation around the grand mean splits exactly: SST = SSB + SSW, where SSB = m sum (mean_j - grand mean)^2 is the between-group part and SSW = sum (x_ji - mean_j)^2 the within-group part. Dividing by the degrees of freedom gives mean squares MSB = SSB / (k - 1) and MSW = SSW / (N - k).

The F test

If every group has the same true mean and the errors are independent normal with equal variance, F = MSB / MSW follows the F distribution with (k - 1, N - k) degrees of freedom, with density f(x) = Gamma((d1+d2)/2) / (Gamma(d1/2) Gamma(d2/2)) (d1/d2)^(d1/2) x^(d1/2 - 1) (1 + d1 x / d2)^(-(d1+d2)/2). The p-value is the upper-tail probability, computed here from the regularised incomplete beta function, p = I_{d2/(d2 + d1 F)}(d2/2, d1/2). When there is no real effect the p-value is uniformly distributed, so p < 0.05 happens in 5 % of experiments: that is the type I error rate and the page checks it by simulation.

What F measures

F is a signal-to-noise ratio: the spread of the group means (scaled by the group size) against the spread of the samples around their own means. A larger spacing between true means, a smaller sigma or more samples all raise F. The share of variance explained is eta^2 = SSB / SST. For two groups F equals the square of the two-sample t statistic.

Simplifications

Equal group sizes, equal variances and normal errors; the test says only that at least one mean differs, not which. A significant p-value is not a measure of effect size, and watching p while samples arrive (as the page does) is exactly the "peeking" that inflates false positives in real experiments, so use it as a demonstration, not as a stopping rule.

What to try
  1. Press the no-effect preset and press Restart several times: F stays near 1 and p wanders, falling under 0.05 about one run in twenty.
  2. Choose the strong-effect preset: the rings separate, SSB dominates and p collapses towards zero as samples arrive (graph 3).
  3. Choose two groups and compare F with the square of the visible difference of means divided by its standard error.
  4. Choose the noisy preset: the beads in each tube overlap, SSW is large and the F ratio stays small.
  5. Choose many samples with a tiny spacing: p can become small although the share of variance explained, eta squared, is tiny.
F distribution under no effect with the 5 % critical region
Sums of squares
p-value as samples accumulate
Samples and group means with standard errors