Thompson Sampling: Bayesian Multi-Armed Bandit

Thompson Sampling is one of the oldest and still best-performing algorithms for the exploration/exploitation trade-off at the heart of reinforcement learning. Instead of a fixed exploration rate like ε-greedy or a hand-tuned confidence bound like UCB1, it keeps a full Bayesian posterior — a Beta(α, β) distribution — over each option's unknown win rate, draws one random sample from every posterior each round, and simply pulls whichever arm's sample is highest. This simulator renders those posteriors as live 3D density curves above each slot machine: watch them start flat and uncertain, then narrow and separate as rounds accumulate, while cumulative reward and regret track how quickly the agent locks onto the best arm.