A Gaussian Mixture Model explains the data as a weighted sum of K Gaussian components. Expectation-Maximization fits it by alternating two steps until the likelihood stops improving:
E-step (soft assignment):
r_ik = π_k N(x_i | μ_k, Σ_k) / Σ_j π_j N(x_i | μ_j, Σ_j)
M-step (refit each component):
N_k = Σ_i r_ik
μ_k = (1/N_k) Σ_i r_ik x_i
Σ_k = (1/N_k) Σ_i r_ik (x_i-μ_k)(x_i-μ_k)^T
π_k = N_k / N
- r_ik is the "responsibility" of component k for point i — a probability, not a hard label. In soft-blend mode a point's color is the weighted blend of all its responsibilities; switch to hard-assign to see the same points snap to their single most-likely component instead.
- Ellipses are the 2σ contour of each component's covariance Σ_k — orientation and stretch come from the eigenvectors/eigenvalues of that 2×2 matrix, so a component can tilt and elongate to fit anisotropic clusters, unlike k-means' circular boundaries.
- Log-likelihood Σ_i log(Σ_k π_k N(x_i|μ_k,Σ_k)) is guaranteed to never decrease across an E+M step — watch it climb on the strip chart below the plot; the run is "converged" once its change drops below 1e-4.
- A diagonal regularizer ε·I is added to every Σ_k so a component that collapses onto a single point never produces a singular, non-invertible covariance. Drag the ε slider toward its low end and shrink a cluster with a small K to watch an ellipse nearly collapse before ε rescues it.
- Drag inside the plot to pan, scroll to zoom — useful once you push N and K up and the blobs get crowded.
Unlike k-means (hard, spherical clusters) or DBSCAN (density contours, no explicit shape), a GMM gives every cluster its own ellipsoidal shape and a genuine probability of membership — the same soft-clustering idea underlies speaker diarization, background subtraction in video, and the E-step used inside many other latent-variable models.