Supervised, Unsupervised and Reinforcement Learning Compared (2D)

Training

Training speed 1.0×

Label noise 8%

Reward sparsity 20%

Dataset shape

Live Readout

Epoch

Elapsed

0.0 s

Supervised accuracy

—

k-means inertia

—

RL avg. reward

—

Explore rate ε

—

How it works

All three panels train on the very same underlying point cloud every epoch, so you can compare directly how each paradigm treats identical data with a different information budget.

Supervised — every point carries a color-coded label (a fraction flipped wrong by the label-noise slider). A segmented boundary line is nudged toward the side that reduces misclassifications, a gradient-free hill-climb that stands in for the same objective a logistic classifier minimizes:

for each point p:
  predicted = sign(p.x − boundary(p.y))
  if predicted != trueLabel(p):
    boundary(p.y) += step · sign(trueLabel − predicted)

Unsupervised — no labels at all. Lloyd's k-means algorithm alternates assigning each point to its nearest of 2 centroids, then moving each centroid to the mean of its assigned points:

assign:   cluster(p) = argmin_k ‖p − centroid_k‖²
update:   centroid_k ← mean{ p : cluster(p) = k }
inertia = mean ‖p − centroid_cluster(p)‖²

Reinforcement — an agent hops across a reward grid with no labels and no fixed dataset, only a reward signal (hidden on a fraction of tiles set by the sparsity slider). It learns a state-value estimate by temporal-difference (TD(0)) bootstrapping, and its exploration rate ε decays from 0.7 toward a 0.05 floor as epochs accumulate:

V(s) ← V(s) + α·[ r + γ·V(s′) − V(s) ],  α = 0.15, γ = 0.9
action = random with prob. ε,  else argmax_{s′} V(s′)

Supervised learning needs the most human effort (labeling every example), unsupervised learning needs the least but gives the least control over the output, and reinforcement learning needs neither labels nor a pre-collected dataset — only a way to try actions and observe rewards.

Supervised boundary Unsupervised (k-means) Reinforcement (TD agent)