⛸️ 2D Proximal Policy Optimization: Clipped Surrogate, GAE and Policy Drift on a Slippery Ice Pond
Plan view of the slippery ice pond with value tints, policy arrows and the latest sampled episode for PPO with and without the clip, live return curves against the optimum and six independent numerical checks (exact evaluation, policy-gradient theorem, Monte Carlo, GAE identities, clip bound, KL estimate).
⚙ Under the hood
Plan view of the slippery ice pond with value tints, policy arrows and the latest sampled episode for PPO with and without the clip, live return curves against the optimum and six independent numerical checks (exact evaluation, policy-gradient theorem, Monte Carlo, GAE identities, clip bound, KL estimate).
Proximal Policy OptimizationReinforcement LearningClipped Surrogate ObjectiveGeneralized Advantage EstimationPolicy GradientTrust RegionMachine Learning
2D · HTML5 Canvas 2D · 60 FPS target · runs fully client-side, no install