About SVD Image Compression
Singular Value Decomposition (SVD) factors any m×n matrix M into M = UΣVᵀ, where U and V are orthogonal matrices and Σ is a diagonal matrix of non-negative singular values σ₁ ≥ σ₂ ≥ … ≥ 0. When applied to an image (treated as a pixel-value matrix), the singular values can be thought of as measuring the "importance" of each layer of structure: the first few singular values capture broad shapes and contrast, while later ones encode fine detail and noise. Keeping only the top k singular values gives the best possible rank-k approximation in the sense of minimising the Frobenius norm of the error — a result known as the Eckart-Young theorem (1936).
This simulation decomposes a 64×64 test image into up to 64 singular components. Drag the rank slider to add or remove components and watch the reconstruction quality change in real time. The bar chart below the images shows the singular value spectrum — components to the left of the cut-off line are included; those to the right are discarded.
Frequently Asked Questions
What does SVD actually compute?
SVD decomposes a matrix M into M = UΣVᵀ. The columns of U are the left singular vectors (output directions), the columns of V are the right singular vectors (input directions), and the diagonal entries of Σ are the singular values — the scaling factors. Every matrix has an SVD; unlike eigendecomposition, it works for non-square and non-symmetric matrices.
Why does keeping more singular values give a better image?
Each singular triplet (σᵢ, uᵢ, vᵢ) adds one rank-1 layer σᵢ·uᵢ·vᵢᵀ to the reconstruction. Larger singular values contribute more energy (structure) to the image. As you increase rank k, you capture more of the total Frobenius energy ∑σᵢ², and the approximation error ‖M − Mₖ‖² = σₖ₊₁² + … + σᵣ² shrinks. Smooth images need fewer components than noisy or detailed ones.
What is the compression ratio and how is it calculated?
Storing a rank-k SVD of an m×n image requires k·(m + n + 1) numbers instead of m·n. The compression ratio is therefore k(m+n+1)/(mn). For a 64×64 image with k=8, this equals 8×129/4096 ≈ 0.25, meaning you store only a quarter of the original data. The trade-off is quality: very low ranks lose fine detail.
What is the Eckart-Young theorem?
The Eckart-Young-Mirsky theorem (1936) states that the rank-k SVD approximation Mₖ = Σᵢ₌₁ᵏ σᵢ uᵢ vᵢᵀ is the closest rank-k matrix to M in both the Frobenius norm and the spectral norm. No other rank-k matrix can be closer. This makes SVD the mathematically optimal lossy compression strategy for matrices.
How are singular values related to eigenvalues?
The singular values of M are the square roots of the eigenvalues of MᵀM (or MMᵀ). If M is symmetric positive semi-definite, its singular values equal its eigenvalues. For a general matrix, eigenvalues can be complex while singular values are always real and non-negative. The largest singular value equals the spectral norm (matrix 2-norm) of M.
Why does the noise image compress poorly?
Random noise has all singular values roughly equal — the energy is spread uniformly across all components rather than concentrated in the first few. There is no low-dimensional structure to exploit. Natural images, by contrast, have a steeply falling singular value spectrum because adjacent pixels are correlated and smooth regions dominate. This is why JPEG and other image codecs can achieve high compression on photos but not on random data.
What real-world uses does SVD have beyond images?
SVD is ubiquitous: it is the engine behind PCA in data science, latent semantic analysis in NLP (finding hidden topics in text), collaborative filtering in recommender systems (Netflix Prize), pseudoinverse computation in numerical linear algebra, and control theory (balanced model reduction). In machine learning, the weight matrices of neural networks are sometimes low-rank approximated via SVD to speed up inference.
What is the difference between SVD and eigendecomposition?
Eigendecomposition A = PDP⁻¹ requires A to be square and diagonalisable, and the eigenvectors form the columns of P. SVD M = UΣVᵀ works for any matrix (including non-square), uses two different orthogonal bases (U and V), and always exists. For symmetric positive semi-definite matrices the two decompositions coincide: U = V = eigenvector matrix and Σ = eigenvalue matrix.
How many singular values does an m×n image matrix have?
An m×n matrix has r = min(m, n) non-zero singular values, where r is its rank. A 64×64 image therefore has at most 64 singular values. In practice natural images have effective numerical rank well below 64 because singular values decay rapidly toward zero — the cumulative energy curve flattens quickly, meaning the first ~10–20 components capture 95%+ of total variance.
What is the Frobenius error shown in the statistics panel?
The Frobenius norm ‖A‖_F = √(∑ᵢⱼ aᵢⱼ²) measures the total "size" of a matrix as the root-sum-of-squares of all entries. The Frobenius error displayed here is ‖M − Mₖ‖_F = √(σₖ₊₁² + … + σᵣ²) — the square root of the energy in the discarded singular values. By the Eckart-Young theorem no rank-k approximation can achieve a smaller Frobenius error.