A Markov chain is a stochastic process satisfying the
Markov property: the next state depends only on the current
state, not on history. It is fully described by a
transition matrix P where Pᵢⱼ = P(state j | state i),
with each row summing to 1. Repeated matrix multiplication πP converges
to the stationary distribution π (eigenvector for
eigenvalue 1 of Pᵀ). Applications include PageRank, hidden Markov
models, MCMC sampling, and text generation.
🇺🇦 Українська
Presets
Transition matrix P
Rows must sum to 1 (auto-normalised)
Iteration0
Current state—
Stationary distribution and convergence
For an ergodic (irreducible + aperiodic) chain, the distribution π(t)
= π(0)·Pᵗ converges to a unique stationary distribution π* regardless
of the starting point. This is the basis of the Monte Carlo Markov
Chain (MCMC) technique used in Bayesian statistics, machine learning,
and physics. The convergence rate is determined by the second-largest
eigenvalue of P — the spectral gap: a large spectral gap
means fast mixing.
About the Markov Chain Simulator
This simulator visualises a discrete-time Markov chain as a directed graph of states linked by transition probabilities. The system is defined entirely by a transition matrix P, where each entry Pᵢⱼ gives the probability of moving from state i to state j and every row sums to 1. On each iteration the probability distribution π is advanced by the vector–matrix product π(t+1) = π(t)·P, and the node sizes track the evolving distribution.
You edit the matrix directly in the table (rows are auto-normalised), choose presets such as Weather, PageRank, Gambler and Random, and set how many steps run per animation frame. A single random walker can also be stepped through the chain to show individual sampled trajectories. Such chains underpin PageRank, weather modelling, queueing theory and MCMC sampling across statistics and machine learning.
Frequently Asked Questions
What is a Markov chain?
A Markov chain is a stochastic process that hops between a finite set of states, where the probability of the next state depends only on the current state and not on the earlier history. This memorylessness is called the Markov property. The chain is fully specified by its transition matrix P.
What does the transition matrix do?
Each entry Pᵢⱼ is the probability of moving from state i to state j in one step. Every row must sum to 1 because the chain has to go somewhere. In this simulator you can type any non-negative values into the table and the rows are automatically re-normalised so they remain valid probabilities.
How does the distribution evolve each step?
The simulator keeps a probability distribution π over the states and updates it by the rule π(t+1) = π(t)·P, which is an ordinary vector–matrix multiplication. Repeating this is equivalent to computing π(0)·Pᵗ. The percentages shown on the nodes and bar chart are the current values of this distribution.
What is the stationary distribution?
The stationary distribution π* satisfies π* = π*·P, meaning it stays unchanged under another step of the chain. It is the left eigenvector of P for eigenvalue 1, equivalently the eigenvector of Pᵀ for eigenvalue 1. For an ergodic chain the distribution converges to this unique π* no matter where it started.
What do the four presets show?
Weather is a classic 3-state sunny/cloudy/rainy model. PageRank is a 4-node web graph illustrating how Google ranks pages by stationary probability. Gambler is a gambler's-ruin chain with two absorbing states at $0 and $3. Random generates a fresh chain of 3 to 5 states with randomly drawn, normalised rows.
What does the step walker button do?
The walker is a single token that performs one actual random transition each time you click it, choosing the next state by sampling from the current row of P. Its trajectory illustrates one concrete realisation of the chain, in contrast to the smooth distribution π which represents the average behaviour over infinitely many such walkers.
What does Steps per frame control?
This slider sets how many distribution updates (and walker steps) are applied during each animation frame, from 1 up to 50. A higher value fast-forwards the chain so you can watch it converge to the stationary distribution more quickly, while a value of 1 lets you observe each iteration in detail.
When does a chain converge to a unique stationary distribution?
Convergence to a single stationary distribution from any start is guaranteed when the chain is ergodic, that is irreducible (every state can reach every other) and aperiodic. The gambler's-ruin preset is not ergodic because $0 and $3 are absorbing, so its long-run behaviour depends on the starting state.
What determines how fast it converges?
The convergence, or mixing, rate is governed by the second-largest eigenvalue modulus of P. The gap between 1 and this value is the spectral gap: a large spectral gap means errors shrink quickly and the chain mixes fast, while a small gap means slow convergence. This is why some chains settle in a few steps and others take many.
How is this connected to PageRank and MCMC?
PageRank treats web pages as states and a random surfer as a walker; the importance of a page is its stationary probability. Markov chain Monte Carlo flips the idea around by constructing a chain whose stationary distribution is a target you wish to sample from, then drawing samples by running the chain — a cornerstone of Bayesian statistics and physics.