Robot Arm Reaching: Cross-Entropy Method Policy Search

A 3-joint robot arm has to reach a randomly placed target, but it is never told the inverse-kinematics equations — instead it learns by trial and error using the Cross-Entropy Method, a gradient-free policy-search algorithm used throughout model-based robot learning. Every generation it samples a whole population of candidate joint-angle policies from a Gaussian distribution, scores each one by how close it gets the end effector to the target, keeps only the elite fraction, and re-fits the sampling distribution's mean and spread to those elites. Watch the population of grey ghost arms narrow generation by generation as the blue "current policy" arm walks itself onto the target, and tune population size, elite fraction and the exploration floor to see the classic exploration/exploitation trade-off in optimization.