Central Limit Theorem: Proof and Consequences

1. Statement of the theorem

Let X₁, X₂, …, Xₙ be independent and identically distributed (i.i.d.) random variables with finite mean μ and finite variance σ² > 0. Define the standardised sum:

The Lindeberg-Lévy Central Limit Theorem states that as n → ∞, Zₙ converges in distribution to the standard Normal:

Remarkably, this holds regardless of the shape of the original distribution of Xᵢ — it could be uniform, exponential, a single fair die, or wildly skewed — as long as the variance is finite.

2. Characteristic functions

The proof relies on the characteristic function of a random variable X, defined as φ_X(t) = E[e^itX]. Two facts make it the right tool:

• The characteristic function of a sum of independent random variables is the product of their characteristic functions: φ_X+Y(t) = φ_X(t)·φ_Y(t).
• (Lévy's continuity theorem) Pointwise convergence of characteristic functions to a function continuous at 0 implies convergence in distribution of the corresponding random variables.

This turns a hard problem about distributions into an easy problem about ordinary complex-valued function limits.

3. Proof sketch (Lindeberg-Lévy CLT)

Without loss of generality assume μ = 0 (subtract the mean) and σ² = 1 (rescale). Let Y = X/σ so E[Y]=0, E[Y²]=1. The characteristic function of Y admits a Taylor expansion near t = 0:

The characteristic function of the standardised sum Zₙ = (1/√n) Σ Yᵢ is, using the product rule for independent variables and the scaling property φ_aX(t) = φ_X(at):

Now take n → ∞. This is exactly the classical limit form (1 + x/n)ⁿ → eˣ with x = −t²/2:

But e^−t²/2 is exactly the characteristic function of the standard Normal N(0,1). By Lévy's continuity theorem, since this limit function is continuous at t = 0, Zₙ converges in distribution to N(0,1). ∎

4. Rate of convergence: Berry-Esseen

The CLT says convergence happens eventually — the Berry-Esseen theorem quantifies how fast, using the third absolute moment ρ = E[|Y|³]:

where Φ is the standard Normal CDF and C is an absolute constant (known to satisfy C ≤ 0.4748). The bound shrinks as 1/√n — convergence is guaranteed but slow: to halve the maximum error you need 4× more samples.

5. Consequences for statistics

• Confidence intervals: the sample mean X̄ₙ is approximately N(μ, σ²/n), justifying the familiar 95% CI X̄ₙ ± 1.96·s/√n even when the underlying population is not Normal.
• t-tests and z-tests: hypothesis tests on means rely on the CLT to approximate the sampling distribution of the test statistic, which is why they remain valid for moderately non-Normal data given a reasonably large n.
• Measurement error: if an observed error is the sum of many small independent contributions (instrument noise, rounding, environmental jitter), the CLT explains why measurement errors are so often observed to be approximately Normal in practice.
• Monte Carlo standard error: the error of a Monte Carlo estimate built from n independent samples shrinks as σ/√n — a direct corollary of the CLT applied to the sample mean.

6. JavaScript: dice-sum simulation

Summing a single uniform die (values 1–6, itself far from Normal) already looks strikingly bell-shaped after just a handful of dice:

// Sum of `k` fair dice, repeated `trials` times, binned into a histogram.
function rollDie() { return 1 + Math.floor(Math.random() * 6); }

function diceSumHistogram(k, trials) {
  const hist = {};
  for (let t = 0; t < trials; t++) {
    let sum = 0;
    for (let d = 0; d < k; d++) sum += rollDie();
    hist[sum] = (hist[sum] || 0) + 1;
  }
  return hist;
}

// Compare empirical mean/variance to CLT prediction: mean = k*3.5, var = k*35/12
function cltCheck(k, trials) {
  const hist = diceSumHistogram(k, trials);
  let mean = 0;
  for (const sum in hist) mean += Number(sum) * hist[sum];
  mean /= trials;
  let variance = 0;
  for (const sum in hist) variance += hist[sum] * (Number(sum) - mean) ** 2;
  variance /= trials;
  return { mean, variance, predictedMean: k * 3.5, predictedVariance: k * 35/12 };
}

console.log(cltCheck(10, 100000));
// { mean: ~35.0, variance: ~29.17, predictedMean: 35, predictedVariance: 29.166... }