Momentum Contrast (MoCo): The Negative-Pair Queue

Self-supervised contrastive learning trains an encoder without any labels by pulling together embeddings of two augmented views of the same example (a positive pair) while pushing apart embeddings of everything else (negative pairs) — the InfoNCE objective at the heart of SimCLR, MoCo and CLIP-style pretraining. This simulator visualizes MoCo's specific answer to the "where do the negatives come from" problem: instead of needing an enormous training batch like SimCLR, a slow momentum-averaged key encoder feeds a running FIFO queue of past embeddings that any later training step can draw negatives from. Six colour-coded classes live as unit vectors on a 3D sphere; each training step samples one class, computes the real InfoNCE loss and its tangent-space gradient against the queue, nudges that class's query embedding along the gradient, drifts its momentum key encoder toward the new query position, and enqueues the fresh key — all driven by the temperature, momentum, queue size and step-size controls, with the loss and cosine similarities updating live.