PCA - 3D Projection Studio

Setting up the studio...

Presets

Live readouts

Graphs

Scree: eigenvalues by component

Cumulative explained variance

Reconstruction MSE vs components kept

Numerical checks

running...

How to use

Choose a preset or design a cloud: three standard deviations, three rotation angles, an offset and some outliers. The slider "Rotate into the PC frame" morphs the cloud to its principal coordinates; "Collapse discarded components" flattens it onto the kept subspace and the orange lines show what is lost. Compare the scree, cumulative-variance and reconstruction-error graphs, and switch standardisation on to see the scale trap disappear.

Drag to orbit, wheel to zoom. The studio, axis rods, screen and textures were modelled and baked in Blender; Three.js r160 animates the cloud.

The science

Principal component analysis finds the orthogonal directions along which a cloud varies most. Centre the data (x - mean), form the covariance matrix and diagonalise it:

C = (1/n) sum_i (x_i - m)(x_i - m)^T = V Λ V^T, V^T V = I, Λ = diag(λ1 ≥ λ2 ≥ λ3)

The columns of V are the principal axes and λ_k is the variance of the data along axis k. Scores s_i = V^T (x_i - m) are the coordinates in the new frame; their covariance is exactly Λ, so the scores are uncorrelated. The eigenproblem here is solved with cyclic Jacobi rotations, which is exact to rounding for a symmetric 3x3 matrix.

Dimension reduction. Keeping the first k scores and mapping back gives the best rank-k approximation in the least-squares sense (Eckart-Young). The mean squared reconstruction error is the sum of the discarded eigenvalues: MSE(k) = λ_(k+1) + ... + λ3. The cumulative explained variance (λ1 + ... + λk) / (λ1 + λ2 + λ3) is the usual way to choose k.

Standardising. PCA depends on units. If one variable has a much larger scale it dominates PC1 regardless of structure (the "scale trap" preset). Dividing every variable by its standard deviation first makes PCA operate on the correlation matrix, so each variable has variance 1 and trace(C) = 3.

Limits. PCA is linear and variance-based. Outliers pull the axes toward them (the "outliers" preset), equal eigenvalues leave the axes undetermined (the sphere), and high variance does not always mean useful signal. The ellipsoid drawn is the 2-sigma surface of a Gaussian with the sample covariance; the data here are Gaussian by construction, which is the case where this picture is exact.

Checks. (1) trace(C) equals the sum of eigenvalues. (2) Each eigenpair satisfies C v = λ v. (3) The rank-2 reconstruction error equals λ3. (4) The PC scores have zero off-diagonal covariance. (5) The top eigenvalue matches the designed variance within sampling error for n = 6000. (6) PC1 aligns with the designed long axis. The first four are algebraic identities; the last two compare with the generating model.

Simplifications. Three dimensions, synthetic Gaussian clouds with optional far outliers, population (1/n) covariance, sign of each axis fixed by its largest component.