CartPole REINFORCE: Policy Field in Phase Space (2D)

This is the flat, native-2D counterpart of the 3D CartPole REINFORCE simulator. It runs the identical Barto–Sutton–Anderson cart-pole physics and the identical Monte-Carlo policy-gradient training loop — a linear softmax policy updated after every episode by the true REINFORCE rule, ∇log π(a|s) times the discounted return minus an optional baseline — but instead of animating a 3D cart and pole, it renders the thing the algorithm actually learns: a live heatmap of push-left vs push-right probability over the pole's angle/angular-velocity plane, with the running episode's real state trajectory traced through that same field. Watch the decision boundary sharpen and curl around θ=0 as training proceeds, while the episode-length graph and live stats confirm the agent is actually getting better at balancing.