Robot Arm Reaching: Cross-Entropy Method Policy Search

Policy Search (CEM)

Population size 24

Elite fraction 25%

Exploration floor 8°

Learning speed 4 gen/s

Training Status

Generation

Elites / pop

6 / 24

Best distance

— m

Policy distance

— m

Status

searching…

Legend

Current policy (mean μ)

Best sample this generation

Elite samples (kept)

Discarded samples

How it works

The arm has 3 joint angles θ = (base yaw, shoulder, elbow). There is no analytic inverse-kinematics solve here — instead a policy search algorithm called the Cross-Entropy Method (CEM) treats θ as parameters to optimize by trial and error, exactly like a gradient-free robot-learning controller would when the reward is not differentiable:

θ_i ~ N(μ, σ²),  i = 1..N        (sample a population)
R(θ_i) = -‖FK(θ_i) − target‖    (reward = negative reach error)
elites = top-k θ_i by R(θ_i)     (keep the best fraction)
μ ← mean(elites)
σ ← std(elites) + floor          (re-fit the search distribution)

Population size — how many candidate joint-angle settings are tried each generation.
Elite fraction — the top slice of samples (by reward) used to re-fit μ and σ; a smaller slice converges faster but risks collapsing onto a bad local pose.
Exploration floor — a minimum σ that stops the search distribution from collapsing to zero before it has explored enough (prevents premature convergence).
The blue arm is the current policy mean μ; grey ghost arms are the population; the amber arm is this generation's single best sample.

Real-world relevance: CEM and its relatives (iCEM, MPPI) are used as gradient-free trajectory optimizers inside model-based robot learning — planning reaching, grasping and locomotion actions when the dynamics or contact reward are too discontinuous for backpropagation, and as the sample-selection step inside more sophisticated policy-gradient and evolution-strategy training loops.

End effector Target

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.