🔁 Markov Chain
A Markov chain is a stochastic process satisfying the Markov property: the next state depends only on the current state, not on history. It is fully described by a transition matrix P where Pᵢⱼ = P(state j | state i), with each row summing to 1. Repeated matrix multiplication πP converges to the stationary distribution π (eigenvector for eigenvalue 1 of Pᵀ). Applications include PageRank, hidden Markov models, MCMC sampling, and text generation. 🇺🇦 Українська
Presets
Transition matrix P
Stationary distribution and convergence
For an ergodic (irreducible + aperiodic) chain, the distribution π(t) = π(0)·Pᵗ converges to a unique stationary distribution π* regardless of the starting point. This is the basis of the Monte Carlo Markov Chain (MCMC) technique used in Bayesian statistics, machine learning, and physics. The convergence rate is determined by the second-largest eigenvalue of P — the spectral gap: a large spectral gap means fast mixing.