🧠 Neural Gas

A Self-Organising Map arranges its nodes on a fixed grid (here a square grid up to 16×16) and updates neighbours based on their grid distance to the winning node, which forces the map to preserve the 2D grid topology even as it bends to fit the data. Neural Gas has no grid at all: every node is simply ranked by its distance to the current input point, and update strength falls off with rank rather than spatial position, letting the nodes settle into whatever arrangement best represents the data.

For each sampled data point, the simulation first finds the "best matching unit" (BMU) — the grid node closest to that point — then updates every node i by Δwᵢ = η·h(d)·(x−wᵢ), where h(d) = e^(−d²/2σ²) and d is the grid distance from node i to the BMU. Nodes near the BMU move almost as much as the BMU itself, while distant nodes barely move, which is what lets the whole grid smoothly fold to trace the data manifold.

Both η and σ (or λ for Neural Gas) are multiplied by an exponential decay factor e^(−t·k) as training epochs progress toward the 200-epoch cap, so early updates are large and affect a wide neighbourhood (coarse, global organisation) while later updates are small and local (fine-tuning). Without this annealing schedule the map would either never stabilise or would fail to unfold correctly from its random initial state.

Quantisation error is the average distance from a sample of data points to their nearest node, computed here over 100 points every 5 epochs; a lower value means the nodes sit closer to where the data actually is, so the map represents the distribution more accurately. Watching this number fall during training is a direct, numeric way to confirm the visual impression that the grid is converging onto the data.

A square grid's fixed neighbour connections naturally match a convex, filled region like the Uniform Square preset, but distributions such as the Ring or Half-Moon are curved and hollow, forcing the grid to twist, fold back on itself, or leave nodes stranded in empty space between the arms of the shape. Neural Gas, lacking a rigid topology, typically tracks these awkward shapes more cleanly since its nodes are not constrained to stay connected to fixed neighbours.