Principal Component Analysis (PCA) in 2D: A Visual Guide |…

What Principal Component Analysis (PCA) Is

Principal Component Analysis is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. The first principal component has the largest possible variance, and each succeeding component has the highest variance possible under the constraint that it is orthogonal to the preceding components.

In 2D, PCA aims to find the line (or plane in higher dimensions) that best fits the data points, maximizing the spread of the projected data.

Why It Happens

The reason PCA works is rooted in linear algebra. When you have correlated variables, their covariance matrix has non-zero off-diagonal elements. The eigenvectors and eigenvalues of this matrix represent the directions (principal components) that capture the most variance in the data, with the corresponding eigenvalues indicating how much variance each component explains.

By projecting the original data onto these principal components, we can reduce dimensionality while retaining as much information as possible.

live demo · related simulation● LIVE

Real-World Applications

PCA is widely used in various fields such as image processing, bioinformatics, and finance. For example, in image compression, PCA can be used to reduce the number of pixels while retaining the most important features of an image.

In finance, PCA helps identify underlying factors that drive stock prices or economic indicators.

Common Questions

Who discovered PCA and when? PCA was developed in the early 1900s by Karl Pearson. However, it gained popularity through the work of Harold Hotelling in the 1930s.

Why is PCA useful for data visualization? PCA simplifies high-dimensional datasets into a lower-dimensional space, making them easier to visualize and understand while preserving important patterns.

Frequently asked questions

How does PCA handle non-linear relationships between variables?

PCA assumes linearity in the relationship between variables. For non-linear relationships, more advanced techniques like kernel PCA or other nonlinear dimensionality reduction methods are used.

Can PCA be applied to datasets with many dimensions?

Yes, PCA can handle high-dimensional data effectively by projecting it onto a lower-dimensional space while retaining the most significant variance.

What happens if all variables are uncorrelated?

If all variables in a dataset are uncorrelated, PCA will find that each principal component is just a copy of one of the original variables. In this case, no dimensionality reduction occurs.

Is PCA sensitive to outliers?

PCA can be affected by outliers because it relies on the mean and variance of the data. Outliers can skew the principal components and lead to misleading results.

Try it live

Everything above runs in your browser — open PCA Visualiser (2D) and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.

▶ Open PCA Visualiser (2D) simulation