AI PID Auto-Tuner (2D): SPSA Reinforcement Learning for Process Control

Hand-tuned PID loops are still the backbone of industrial control — but Industry 4.0 plants increasingly hand the tuning itself to an AI agent. This 2D simulator models a real tank-level loop as a first-order-plus-dead-time process driven by a PID valve controller, then lets an SPSA (Simultaneous Perturbation Stochastic Approximation) reinforcement-learning agent search for the gains that minimize a real ISE-plus-overshoot cost function using only two rollouts per update — no gradient of the plant model required, exactly as a model-free industrial auto-tuner would. A live strip chart shows the level step-response settle in faster, tighter, less oscillatory as training runs, while a Kp–Ki gain-space scatter plot shows the exact path the agent walked to get there.