This simulation animates a discrete-time Markov chain on a finite set of states. A random walker hops between states according to a transition matrix P, where each row holds the probabilities of moving to every other state and sums to 1. The state diagram draws nodes in a circle with arrows weighted by probability, while a bar chart compares observed visit frequencies against the stationary distribution π, computed by power iteration of π = πP.
The Preset menu loads ready-made chains (Weather, PageRank, Gambler's Ruin, Hardy-Weinberg, a random 4-state chain), the Step-delay slider sets pace from 50 to 2000 ms, and Step, Run and Reset drive the walk. Markov chains underpin Google's PageRank, speech recognition, queueing models and population genetics, making them one of probability theory's most widely applied tools.
What is a Markov chain?
A Markov chain is a stochastic process that moves between a set of states, where the probability of the next state depends only on the current state and not on the path taken to reach it. This memoryless feature is called the Markov property. Each step is decided by the transition matrix loaded from the chosen preset.
What does the bar chart at the bottom show?
The purple bars show the empirical visit frequencies the walker has actually accumulated, while the gold bars show the theoretical stationary distribution π. As more steps run, the purple bars should converge towards the gold ones, illustrating the long-run behaviour of the chain.
How is the stationary distribution calculated?
The simulation uses power iteration: it starts with a uniform distribution and repeatedly multiplies it by the transition matrix P until the change between iterations falls below 1e-8, up to 2000 iterations. The result is the vector π satisfying π = πP, the chain's equilibrium probabilities.
Step advances the walk by exactly one transition. Run starts continuous stepping at the chosen delay and toggles to Pause. Reset reloads the current preset, clearing the step count and visit history so you can start the random walk afresh.
It sets the time between automatic transitions while running, from 50 ms (fast) to 2000 ms (slow), in 50 ms increments, defaulting to 600 ms. Adjusting it while running restarts the timer at the new pace so you can watch each hop or speed up convergence.
Once more than 20 steps have run, it reports the mean absolute difference between the empirical visit fractions and the stationary distribution π, shown as a percentage. Smaller values mean the random walk is closely matching the theoretical equilibrium.
Gambler's Ruin has two absorbing states, $0 and $4, which the chain can enter but never leave. Such chains do not have a single interior stationary distribution in the usual sense, so the walk eventually gets trapped at one boundary, modelling a gambler going broke or hitting their target.
It models a web surfer randomly following links between four pages. PageRank, the algorithm behind early Google search, treats the web as a giant Markov chain and ranks pages by their stationary probability, that is, how often a random surfer lands on them in the long run.
Yes for the methods shown: transitions are sampled correctly from each row of P using a cumulative-probability draw, and π is found by genuine power iteration. The preset matrices are illustrative examples rather than measured real-world data, so the qualitative behaviour is faithful even if specific numbers are stylised.
Not always. A unique stationary distribution that the walk converges to is guaranteed when the chain is irreducible and aperiodic. Chains with absorbing states, disconnected components or strict periodicity can fail this, which is why presets like Gambler's Ruin behave qualitatively differently from the Weather chain.