Self-Organizing Maps: How Kohonen Networks Learn Without Labels

The core idea and architecture

A SOM is a single layer of neurons arranged on a fixed grid — typically rectangular or hexagonal. Each neuron i carries a weight vector w_i that lives in the same space as the input data: if the inputs are 50-dimensional, every neuron's weight vector is also 50-dimensional, even though the neurons themselves sit on a flat 2D grid. The grid position never changes — only the weight vectors are learned. Two things happen at once: the weight vectors move to cover the data, like clustering, while the fixed grid imposes a topology that forces neighbouring neurons to learn similar things. The trained grid becomes a low-dimensional, topology-preserving picture of a high-dimensional dataset — loosely modelled on cortical maps in the brain, such as tonotopic maps in the auditory cortex, where nearby neurons respond to nearby stimuli.

Competitive learning and the Best Matching Unit

Training is driven by competition. For each input vector x, every neuron measures how close its weight vector is to the input, and the closest neuron wins — the Best Matching Unit (BMU):

c = argmin_i ‖x − w_i‖,  ‖x − w_i‖ = √(Σ_j (x_j − w_ij)²)

h_ci(t) = exp(−d(c,i)² / (2σ(t)²))    // Gaussian neighborhood, shrinks with time
w_i(t+1) = w_i(t) + α(t)·h_ci(t)·(x(t) − w_i(t))

In plain "winner takes all" competitive learning, only the BMU would update. A SOM's crucial addition is that the BMU and its neighbours on the grid all move toward the input, controlled by the neighborhood function h_ci, which shrinks over time — this is what stitches the map together so adjacent neurons end up representing adjacent regions of the data. Early in training the radius σ is large (the ordering phase, where the map unfolds to match the broad shape of the data); as σ shrinks, updates become local (the convergence phase), letting each neuron specialise.

The training algorithm, step by step

Repeated for many iterations: initialise each neuron's weight vector (randomly or from principal components for faster convergence); present a randomly chosen input x; find the BMU — the neuron whose weights are closest to x; update the BMU and its neighbours toward x; decay the learning rate α and neighborhood radius σ; repeat until the weights stabilise. Because each step only requires a distance computation and a local update, the SOM scales well and is easy to implement, which is part of why it remained popular decades after its introduction.

Reading a trained map: the U-matrix

Once trained, the SOM is a layout — but how do you see clusters in it? The standard tool is the U-matrix (unified distance matrix): for each neuron it computes the average distance between that neuron's weight vector and its immediate grid neighbours' weight vectors. Low U-value means a neuron is similar to its neighbours — inside a cluster; high U-value means a big jump in the data — a boundary between clusters. Rendered as a heat map, the U-matrix shows clusters as low "valleys" separated by high "ridges," turning an abstract high-dimensional dataset into a readable terrain map.

Applications

SOMs are valued wherever exploratory visualisation of complex data matters: document and text mining (Kohonen's own WEBSOM project organised millions of documents into a browsable 2D map); bioinformatics (clustering gene-expression profiles); economics (the "poverty map of the world," arranging countries by welfare indicators onto a single grid); fault detection (normal operating states map to one region, deviations land elsewhere); and colour quantisation (compressing a palette while preserving perceptual relationships). The SOM is simultaneously a clustering method, a dimensionality-reduction method, and a visualisation method — that combination is why it remains a teaching staple and a practical tool decades after Kohonen first described it.

Frequently asked questions

Try it live

Everything above runs in your browser — open Self-Organising Map (SOM) and watch a Kohonen map unfold over data and form clusters in real time. Nothing is installed, nothing is uploaded.