Gaussian Mixture Models: Soft Clustering with EM

Mixture Model

Components K 3

Data points 90

Auto-step speed 2.0/s

Iteration 0 — press Play or Step

EM Progress

Iteration

Log-likelihood

—

Δ log-lik

—

Mean entropy

—

Components

How it works

A Gaussian Mixture Model explains the data as a weighted sum of K 2D Gaussians. Unlike k-means, points are never forced into one cluster — every point gets a soft responsibility for each component, and that responsibility is what shapes the coloured blend and the density surface below.

p(x) = Σₖ wₖ · N(x | μₖ, Σₖ)
N(x|μ,Σ) = 1/(2π√|Σ|) · exp(-½ (x-μ)ᵀ Σ⁻¹ (x-μ))

E-step — with the current parameters fixed, compute how responsible each component is for each point:

rᵢₖ = wₖ N(xᵢ|μₖ,Σₖ) / Σⱼ wⱼ N(xᵢ|μⱼ,Σⱼ)

M-step — with responsibilities fixed, re-estimate every component from its weighted share of the data:

Nₖ = Σᵢ rᵢₖ
μₖ  = (Σᵢ rᵢₖ xᵢ) / Nₖ
Σₖ  = (Σᵢ rᵢₖ (xᵢ-μₖ)(xᵢ-μₖ)ᵀ) / Nₖ
wₖ  = Nₖ / N

Each E/M pair can only raise (never lower) the data log-likelihood, so the algorithm climbs monotonically to a local optimum — watch the Δ log-lik readout shrink toward zero as it converges. Mean entropy tracks how "soft" the assignment still is: high near overlapping clusters, near zero once a point is confidently claimed by one component.

Components K — how many Gaussians the model is allowed to fit; the true data was generated from an unknown number of clusters, so K too small or too large is itself an instructive mismatch.
Reinit means — restarts EM from a fresh random initialisation on the same data, showing how the local optimum reached can depend on the starting point.
New data — resamples the underlying dataset from new hidden Gaussian sources.

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.

Gaussian Mixture Models: Soft Clustering with Expectation-Maximization