Hindsight Experience Replay: Robot Reaching with Sparse Rewards

Two identical 2-link robot arms try to reach randomly placed targets using the same sparse binary reward and the same REINFORCE policy-gradient optimizer. The only difference: the blue arm applies Hindsight Experience Replay, relabeling every failed attempt's actual end position as if it had been the intended goal, so it always has a successful trajectory to learn from. The orange arm gets no such help — it only ever learns from the rare episodes where it happens to land on the real goal by chance. Watch the live success-rate chart and the growing gap between the two curves, exactly the effect that makes HER a standard technique for training real robotic manipulators under sparse, goal-based rewards.