3D Robot Arm: Q-Learning Reaching Simulator

This simulator trains a two-joint robot arm to reach a target with real tabular Q-learning over discretized joint angles: shoulder and elbow each split into twelve bins form a 144-state table, four actions nudge one joint at a time, and every step applies Q(s,a) += α·(reward + γ·max(Q(s′,a′)) − Q(s,a)) based on how much closer the end-effector moved. A live 3D bar chart floating beside the arm renders the learned Q-table in real time, so you can watch the agent's confidence about each joint configuration grow as episodes accumulate. Tune the learning rate, discount factor and exploration decay to see how each reshapes convergence.