3D Bayesian vs Frequentist Coin Lab: Posterior Landscape and Interval Coverage Racks

Setting up the lab...

Presets

Live readouts

Graphs

Exact coverage against the true theta (dashed green: nominal level; vertical line: your coin)

Intervals as the flips arrive: frequentist (orange) and credible (blue); green line = truth

p-value (orange), P(null) from the Bayes factor at equal prior odds (purple), P(theta > theta0 | data) (blue)

Coverage of each method: simulated (big dot) against the exact binomial sum (white dot)

Try this

Tiny sample + extreme coin: watch the Wald rack turn red. Wrong strong prior: see the posterior refuse to move. Lindley paradox: significant p, yet BF01 above 1.

Numerical checks

running...

How to use

Flips arrive one by one and every number updates. Pick a preset or move any slider. Drag to orbit, wheel to zoom; the surface height is the posterior density at each flip count (rows) and theta (columns). Change theta0 with the slider. The two racks show the same experiment repeated many times: each bar is an interval from a fresh data set, red bars miss the true value (green line). Compare how often each method misses with the nominal level. Space pauses, R restarts with a new random data set.

The bench, frames, racks and coin were built in Blender; all dynamics run in JavaScript.

The science

The setup. A coin lands heads with an unknown probability theta. After n flips we see k heads, so the likelihood is binomial: P(k | theta) = C(n,k) theta^k (1-theta)^(n-k)

Frequentist view. theta is a fixed number; the data are random. A 95 % confidence interval is a recipe that, over many repeated experiments, contains the true theta in at least (or about) 95 % of them. The Wald interval p +/- z sqrt(p(1-p)/n) uses a normal approximation; the Wilson interval inverts the score test; the Clopper-Pearson interval inverts the exact binomial test, [ Beta^-1(a/2; k, n-k+1), Beta^-1(1-a/2; k+1, n-k) ], and is guaranteed to cover at least the nominal level (often more). The p-value is the exact two-sided binomial probability of an outcome no more likely than the observed one under theta0.

Bayesian view. theta has a probability distribution that expresses what we know. Start with a prior Beta(a0, b0); Bayes' rule gives a posterior that is again a Beta (conjugacy): posterior(theta | k, n) = Beta(a0 + k, b0 + n - k), mean = (a0 + k) / (a0 + b0 + n) The 95 % credible interval is the central region holding 95 % of that posterior mass; it is a direct probability statement about theta given this prior and these data. A Bayes factor compares the point null theta = theta0 with a uniform alternative: BF01 = P(k | theta0) (n + 1). For a flat prior and small samples the posterior is pulled toward 0.5 relative to k/n; with lots of data the prior is forgotten and credible and confidence intervals agree.

What the graphs show. The coverage graph is not simulated: for each true theta it sums the binomial probabilities of every outcome k whose interval contains theta. It shows the famous sawtooth of discrete data and the poor Wald coverage near 0 and 1. The credible interval is built to be right on average over the prior, not at every fixed theta, so its coverage at a given theta can be above or below 95 %. The Lindley paradox preset shows data that reject the null at p = 0.05 while the Bayes factor mildly favours it.

Honest simplifications. (1) One parameter only, a coin. (2) The credible interval is equal-tailed, not highest-density. (3) The Bayes factor assumes a uniform alternative; other alternatives change it. (4) Both views are valid answers to different questions; the page does not declare a winner. (5) The "random" flips come from a seeded generator, so the Restart button gives a new data set with the same settings.

Checks run in the page. A Beta density integrates to 1 and has the right mean (midpoint rule, independent of the incomplete beta code); the binomial CDF from the incomplete beta equals the direct sum; simulated coverage of the Clopper-Pearson and credible intervals matches the exact enumeration within four standard errors; the lowest Clopper-Pearson coverage does not fall below the level; at n = 4000 the credible and Wald intervals have the same width.