Variational Inference: Watching a Posterior Get Approximated (2D)

Instead of sampling a posterior distribution point by point the way MCMC does, variational inference turns Bayesian inference into an optimization problem: fit a simple distribution q(θ) — here a diagonal Gaussian — to the true posterior by maximizing the Evidence Lower Bound (ELBO). This 2D counterpart renders the true log-density directly as a filled contour plot and runs the same genuine stochastic gradient ascent on the ELBO as the 3D version, using the reparameterization trick to differentiate through Monte-Carlo samples, exactly as libraries like Pyro or Stan's ADVI do. Switch between a single correlated Gaussian mode, a bimodal target and a curved "banana" posterior to see variational inference's real behavior: fast convergence, but mode-seeking collapse whenever the true posterior has more structure than the Gaussian family can represent — and watch the ELBO trace climb (or plateau at a discouragingly low value) as it happens.