Bayesian Updating: Watching a Belief Sharpen in Real Time

Updating a belief, not just a number

Flip a coin of unknown bias p and every outcome should nudge your belief about p. Bayes' rule makes that nudge precise: your posterior belief after seeing data is proportional to your prior belief times how likely that data was under each possible value of p.

P(p | data)  ∝  P(data | p) · P(p)

posterior   ∝   likelihood   ×   prior

The proportionality hides a normalising constant — the probability of the data averaged over every possible p — but for the coin-flip problem there is a shortcut that avoids computing it at all.

Why Beta and Binomial fit together

Model each flip as a Bernoulli trial with unknown success probability p, and choose a Beta(α, β) distribution as the prior over p. The Beta distribution is defined on exactly [0, 1], which matches a probability, and its density is proportional to p^(α-1)·(1-p)^(β-1) — algebraically the same shape as the binomial likelihood for k successes in n trials, p^k·(1-p)^(n-k). Multiplying a Beta prior by a binomial likelihood therefore just adds exponents, and the result is another Beta distribution:

prior:      p ~ Beta(α, β)
data:       k successes in n flips
posterior:  p ~ Beta(α + k, β + n - k)

This is a conjugate prior: the posterior stays in the same family as the prior, so updating never requires numerical integration — just add the successes to α and the failures to β. Interpret α and β as pseudo-counts: a Beta(1, 1) prior (uniform over [0,1], total weight 2) encodes complete ignorance, and after observing 7 heads and 3 tails the posterior Beta(8, 4) behaves exactly as if you had started with 1 fictitious head and 1 fictitious tail already in hand.

Point estimates and credible intervals

A Beta(α, β) distribution has mean α/(α+β) and mode (α-1)/(α+β-2) for α, β > 1, so the posterior mean after k heads in n flips with a uniform prior is (k+1)/(n+2) — Laplace's classic rule of succession, which sensibly avoids ever returning exactly 0 or 1 even after an unbroken run. A 95% credible interval — the direct Bayesian analogue of a confidence interval, but interpretable as "there is a 95% probability the true p lies in this range given the data" — is read straight off the Beta posterior's quantiles, and it visibly narrows as n grows, which is the update you watch happen live in the demo.

Sequential updating and the order-independence property

Because the posterior after n flips only depends on the total count of heads and tails, not their order, you can update one flip at a time and get exactly the same final posterior as updating once on the whole batch — today's posterior becomes tomorrow's prior, forever. This is why Bayesian updating is naturally sequential: no need to store the raw data, just the running (α, β) pair, which is also why it is the standard engine behind multi-armed bandits, spam filters and A/B tests that need to keep learning as new observations arrive.

Choosing a prior, and why it eventually stops mattering

A strong prior (large α+β) needs more data to move; a weak or flat prior (α=β=1) is swayed almost immediately by the first few flips. As n grows, the likelihood's exponents k and n−k dominate the fixed α and β, and posteriors from very different starting priors converge to the same answer — a version of the Bernstein-von Mises theorem. The prior mostly matters when data is scarce, which is precisely when honestly encoding what you already believe is most valuable.

Frequently asked questions

Try it live

Everything above runs in your browser — open Bayesian Updating and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.