Statistics ★★☆ Medium

📊 PCA & SVD Visualiser

Generate a 2D point cloud, drag individual points, or adjust the covariance sliders. The simulation computes the covariance matrix, finds its eigenvectors in real time, and draws the principal component axes scaled by standard deviation. A bar chart shows explained variance per component.

Dataset

Display

Explained Variance

PC1
80%
PC2
20%

Covariance Matrix

⎡           ⎤
⎣           ⎦
λ₁ (PC1)—
λ₂ (PC2)—
Condition #—
C = XᵀX / (n−1)
C v = λ v (eigen)
EVR = λₖ / Σλⱼ
Click canvas to add pts

About PCA & SVD

Principal Component Analysis

PCA finds the orthogonal directions of maximum variance in a dataset. Given n data points in d dimensions (zero-mean), the covariance matrix C = XᵀX/(n−1) is real symmetric and diagonalisable: C = Q Λ Qᵀ, where the columns of Q are eigenvectors (principal components) and Λ = diag(λ₁, …, λ_d) with λ₁ ≥ λ₂ ≥ … Project onto the first k columns of Q to reduce dimensionality while retaining the most variance.

Singular Value Decomposition

Any m×n matrix M = U Σ Vᵀ, where U (m×m) and V (n×n) are orthogonal and Σ is diagonal with non-negative entries σ₁ ≥ σ₂ ≥ … (singular values). PCA of X is equivalent to SVD of X/√(n−1): the principal components are the columns of V and the singular values relate to eigenvalues by σₖ = √((n−1) λₖ). SVD is numerically more stable than eigendecomposing C directly.

Geometric Interpretation

In 2D, PCA finds the major axis of the data ellipse (PC1, pointing along maximum spread) and the minor axis (PC2, perpendicular). The ratio σ₁/σ₂ equals the ratio of semi-axes. The explained variance ratio EVR₁ = λ₁/(λ₁+λ₂) is the fraction of total spread captured by PC1. For a sphere of uncorrelated data EVR₁ = 50%; for a highly elongated ellipse EVR₁ → 100%.

Applications

PCA and SVD are workhorses of data science and physics: image compression (truncated SVD), face recognition (eigenfaces), genomics (population structure), finance (factor models), spectroscopy (multivariate curve resolution), and dimensionality reduction before clustering. In quantum mechanics, the reduced density matrix for a bipartite system is diagonalised by the Schmidt decomposition — the quantum analogue of SVD.

About the PCA & SVD Visualiser

This simulation shows how Principal Component Analysis discovers the directions of greatest variance in a 2D point cloud. From the generated points it computes the sample mean, builds the 2×2 covariance matrix C = XᵀX/(n−1), and solves its characteristic equation analytically to obtain the two eigenvalues λ₁ ≥ λ₂ and their orthogonal eigenvectors. Those eigenvectors are the principal component axes drawn on the canvas.

The sliders set the number of points, the spreads σₓ and σᵧ, the rotation θ that correlates the axes, and Gaussian noise, so you can watch the covariance matrix and explained-variance bars update live. PC1 follows the long axis of the data ellipse, PC2 the short one. The same maths underpins dimensionality reduction, image compression via truncated SVD, eigenfaces and factor models in finance.

Frequently Asked Questions

What is this simulation showing?

It generates a cloud of 2D points and applies Principal Component Analysis to it. The principal component axes (PC1 and PC2) are drawn through the data mean, scaled by standard deviation, and a bar chart reports how much of the total variance each component explains.

How are the principal components actually computed?

The page centres the data, forms the 2×2 covariance matrix C = XᵀX/(n−1), and finds its eigenvalues from the quadratic trace/determinant formula. Each eigenvector is then read off in closed form, so PC1 and PC2 are exact rather than iteratively estimated.

What do the dataset sliders do?

N points sets how many samples are drawn; σₓ and σᵧ set the spread along the two underlying axes; Rotation θ tilts that frame to introduce correlation between x and y; and Noise adds extra isotropic Gaussian jitter. Changing any slider regenerates the cloud and recomputes the PCA.

What does explained variance ratio mean here?

The explained variance ratio for a component is its eigenvalue divided by the sum of eigenvalues, EVRₖ = λₖ/Σλⱼ. PC1's bar shows the fraction of total spread captured by the longest axis. For a circular cloud both bars sit near 50%; for a very elongated ellipse PC1 approaches 100%.

Why is the 2σ ellipse drawn the way it is?

The dashed ellipse is the contour two standard deviations from the mean along each principal axis. Its semi-axes are 2√λ₁ and 2√λ₂, and it is rotated to align with the eigenvectors. For Gaussian data it encloses roughly 86% of the points, giving a visual summary of the covariance.

What is the condition number telling me?

The condition number shown is the ratio of the two eigenvalues, λ₁/λ₂. A large value means the data is highly stretched and the covariance matrix is close to singular, so the minor direction carries little information. When PC2 collapses to nearly zero the readout shows infinity.

How does SVD relate to the PCA shown?

PCA of X is equivalent to the singular value decomposition of X/√(n−1). The principal components are the right singular vectors and the singular values satisfy σₖ = √((n−1)λₖ). Computing SVD directly on the data is numerically more stable than eigendecomposing the covariance matrix, though both give the same axes.

Can I edit the data myself?

Yes. Click anywhere on the canvas to add an individual point, press Regenerate to draw a fresh random cloud from the current slider settings, or use Clear to empty the canvas. The PCA recomputes after every change, so you can build a custom distribution and see its principal axes immediately.

Is the eigenvalue calculation physically accurate?

For a 2×2 symmetric matrix it is exact. The eigenvalues come from λ = tr/2 ± √(tr²/4 − det), which is the analytic solution of the characteristic polynomial. Both eigenvalues are clamped to be non-negative, matching the fact that a real covariance matrix is positive semi-definite.

What does projecting onto a component mean?

Enabling projections draws a line from each point to its foot on the PC1 axis, illustrating dimensionality reduction. Keeping only the PC1 coordinate discards the PC2 information; the squared length of those dropped segments equals the reconstruction error, which PCA minimises for any chosen number of components.

Where is PCA used in the real world?

PCA and SVD power image compression, face recognition through eigenfaces, population-structure analysis in genomics, factor models in finance, and noise reduction before clustering or regression. In quantum mechanics the closely related Schmidt decomposition diagonalises a bipartite reduced density matrix.