Bayesian Inference: Prior, Likelihood & Posterior Explained

The product rule, twice over

Everything starts from the definition of conditional probability: P(A|B) = P(A ∩ B) / P(B). The same joint probability P(H ∩ E) can be written two ways — as P(E|H)·P(H) or as P(H|E)·P(E) — and setting these equal gives Bayes' theorem:

P(H|E) = P(E|H) · P(H) / P(E)

H = hypothesis (what you care about)
E = evidence (what you observed)
P(E) = Σᵢ P(E|Hᵢ)·P(Hᵢ)   ← normalising constant

That is the whole theorem — the product rule applied twice. Its power comes entirely from choosing what to condition on: H is the hypothesis (a model, a diagnosis, a coin's bias), E is the evidence you actually observed.

Prior, likelihood, posterior

The prior P(H) is what you believed before the data. The likelihood P(E|H) is how probable the evidence is if H is true — what the model predicts. The posterior P(H|E) is your updated belief, and it becomes the prior for the next observation. The denominator P(E), the marginal likelihood, is the same for every hypothesis and simply normalises the posterior to sum to 1.

live demo · Beta-Binomial posterior narrowing with each observation● LIVE

The base-rate fallacy: a positive test isn't proof

A disease affects 1% of the population. A test is 99% sensitive and 98% specific. Given a positive result, what's the chance of actually having the disease?

P(+) = 0.99×0.01 + 0.02×0.99 = 0.0297
P(D|+) = 0.99×0.01 / 0.0297 ≈ 0.333  → only 33%!

Despite a highly accurate test, one positive result means just a 33% chance of disease, because the disease is rare — the low prior dominates. Test again and use 0.33 as the new prior: the second positive pushes P(D|++) to about 96%. This is exactly the point of sequential Bayesian updating, and it is why ignoring the prior — the base-rate fallacy — produces wildly overconfident diagnoses.

Conjugate priors: updates without integration

When the prior and posterior belong to the same distributional family, the prior is called conjugate to the likelihood, and updating needs no numerical integration at all. Flip a coin n times, observe k heads, and if the prior on bias θ is Beta(α, β), the posterior is simply Beta(α + k, β + n − k). A uniform prior (α = β = 1) gives posterior mean (k+1)/(n+2) — Laplace smoothing. Other classic pairs: Gaussian likelihood + Gaussian prior → Gaussian posterior (the linear Kalman filter); Poisson likelihood + Gamma prior → Gamma posterior; Categorical + Dirichlet → Dirichlet (language models).

Naive Bayes: prior × likelihood, one word at a time

A spam filter is Bayes' theorem applied word by word: P(spam|words) ∝ P(spam) × Π P(word|spam), assuming word occurrences are conditionally independent given the class — the "naive" assumption. With Laplace smoothing to avoid zero probabilities for unseen words, this simple model classifies "free prize money claim" as spam and "office meeting agenda review" as ham using nothing more than word-count ratios learned from a small training set.

Frequently asked questions

What is the difference between prior, likelihood and posterior?

The prior P(H) is what you believed about a hypothesis before seeing data. The likelihood P(E|H) is how probable the observed evidence is if the hypothesis were true. The posterior P(H|E) is your updated belief after combining the two via Bayes' theorem — and it becomes the new prior for the next observation.

Why does a positive medical test not mean you have the disease?

This is the base-rate fallacy. Even a test with 99% sensitivity and 98% specificity, applied to a disease with 1% prevalence, gives only a 33% chance of actually having the disease given one positive result — because the low prior probability of the disease dominates the calculation. Bayes' theorem makes this explicit where intuition usually fails.

What makes a prior 'conjugate' to a likelihood?

A prior is conjugate to a likelihood when the posterior belongs to the same distributional family as the prior, allowing closed-form updates with no numerical integration. The classic example is a Beta prior with a Binomial likelihood, which always yields a Beta posterior — simply add the observed successes and failures to the prior's parameters.

Try it live

Everything above runs in your browser — open Bayesian Inference and watch a Beta-Binomial posterior update in real time as you add observations, with the 95% credible interval shifting after every data point.

▶ Open Bayesian Inference simulation