Variational Inference: Watching a Posterior Get Approximated

Instead of sampling a posterior distribution point by point the way MCMC does, variational inference turns Bayesian inference into an optimization problem: fit a simple distribution q(θ) — here a diagonal Gaussian — to the true posterior by maximizing the Evidence Lower Bound (ELBO). This simulator renders a real 2D parameter space as a 3D log-density landscape and runs genuine stochastic gradient ascent on the ELBO, using the reparameterization trick to differentiate through Monte-Carlo samples, exactly as libraries like Pyro or Stan's ADVI do. Switch between a single correlated Gaussian mode, a bimodal target and a curved "banana" posterior to see variational inference's real behavior: fast convergence, but mode-seeking collapse whenever the true posterior has more structure than the Gaussian family can represent.