t-SNE Explained: How Perplexity Shapes the Clusters You See

Why PCA alone isn't enough

Principal Component Analysis finds the directions of greatest variance in a dataset and projects the data onto them. It is fast, deterministic, and the axes it produces are interpretable — you can look at the loadings and say exactly which original features drive each component. But PCA is fundamentally a linear technique: it can only capture straight-line relationships between variables. When customer segments, gene-expression clusters, or handwritten-digit classes sit on curved, tangled manifolds in high-dimensional space, a linear projection often smears them together even though the underlying groups are perfectly separable. This is the gap that t-Distributed Stochastic Neighbor Embedding (t-SNE) was designed to fill: instead of preserving global variance, it preserves local neighbourhood structure, so points that were close together in the original space stay close in the 2D map.

The core idea: matching probability distributions

t-SNE starts by converting distances between points in the original high-dimensional space into conditional probabilities that represent similarity: for each point, it asks, 'if I picked a neighbour proportional to a Gaussian centred on this point, how likely is each other point to be chosen?' This turns the geometry of the dataset into a probability distribution over point pairs. t-SNE then tries to find a low-dimensional (typically 2D) arrangement of points whose pairwise similarities — measured with a heavier-tailed Student-t distribution rather than a Gaussian — reproduce that same probability distribution as closely as possible. The heavy tail is the clever part: it lets moderately distant points in the high-dimensional space repel each other more strongly in the 2D map, which is what prevents everything from collapsing into a single crowded blob and instead produces the visually distinct 'islands' t-SNE is famous for.

The optimisation minimises the Kullback-Leibler divergence between the high-dimensional and low-dimensional similarity distributions using gradient descent, iterating for hundreds or thousands of steps until the layout stabilises.

Perplexity: the one parameter that changes everything

Perplexity is roughly interpreted as the effective number of neighbours each point considers when building its local similarity distribution — a common working range is 5 to 50, with 30 being a frequent default. Set it too low, and the algorithm treats each point as having only a handful of neighbours, producing many small, fragmented clusters that may just be noise. Set it too high, and distant points get pulled into the same neighbourhood calculation, blurring genuine cluster boundaries into a single diffuse shape. Because a dataset of a few hundred rows and a dataset of a hundred thousand rows need very different perplexity values to reveal the same structure, there is no universally correct setting — it has to be tuned by eye, often by running the algorithm multiple times and comparing.

What t-SNE plots can and cannot tell you

This is the most important caveat for anyone reading a t-SNE scatterplot: the distances between clusters, and the sizes of the clusters themselves, are not reliably meaningful. Because the heavy-tailed low-dimensional distribution and the iterative optimisation reshape the geometry non-linearly, two clusters that appear far apart might be more or less related than two that appear close, and a tight-looking cluster in the plot doesn't necessarily correspond to a tightly-packed group in the original space. What t-SNE reliably tells you is which points belong together — the local neighbourhood structure — not how those neighbourhoods relate to each other globally. Random initialisation also means that running t-SNE twice on the same data with different seeds can produce visually different layouts, even though the local groupings are similar, so a single run should never be treated as a definitive picture.

t-SNE versus PCA in practice

The two techniques are complementary rather than competing. PCA is fast (it scales to very large datasets and can be computed in closed form via singular value decomposition), and its axes are interpretable through the component loadings, which makes it well suited to feature reduction before feeding data into a downstream model. t-SNE is far more computationally expensive, its axes have no interpretable meaning, and it is not designed to transform new, unseen data points the way PCA's fitted transformation can. But t-SNE routinely produces visibly tighter, more separated clusters on the same dataset because it is explicitly optimising for local neighbourhood preservation rather than global variance capture. A common workflow uses PCA first to cut a dataset from dozens of dimensions down to 30-50, then applies t-SNE on that reduced representation — this speeds up t-SNE considerably and denoises the input before the neighbourhood calculations begin.

Practical tuning tips

Beyond perplexity, two other settings matter. Learning rate controls the step size of the gradient descent; too high and points fly apart into a uniform ball, too low and the optimisation gets stuck in poor local layouts (a rule of thumb is to scale the learning rate with dataset size). The number of iterations needs to be large enough for the layout to stabilise — cutting it short leaves clusters only partially formed. It's also worth running t-SNE with several different perplexity values and random seeds side by side, since a pattern that appears consistently across multiple runs is far more trustworthy than one that shows up in a single plot.