loading...
running...
The science
t-SNE turns high-dimensional similarities into a low-dimensional map by matching two probability distributions.
1. Similarities in the data. For each sample i a Gaussian is centred on it and its width σ_i is found by bisection so that the Shannon entropy of the neighbour distribution gives the chosen perplexity:
p(j|i) = exp(-|x_i - x_j|^2 / 2σ_i^2) / Σ_(k≠i) exp(-|x_i - x_k|^2 / 2σ_i^2), Perp = 2^H(P_i) = e^H(P_i) with H in nats
Perplexity is therefore a smooth count of effective neighbours (typically 5-50). The distribution is symmetrised, p_ij = (p(j|i) + p(i|j)) / 2n, so that Σ p_ij = 1 and an isolated sample still contributes.
2. Similarities in the map. Map points y_i use a Student-t kernel with one degree of freedom (heavy tails), which lets moderately distant points sit further apart and relieves the crowding problem:
q_ij = (1 + |y_i - y_j|^2)^-1 / Σ_(k≠l) (1 + |y_k - y_l|^2)^-1
3. Objective and gradient. The map minimises the Kullback-Leibler divergence, which is not symmetric and punishes placing neighbours far apart much more than placing strangers close together:
KL(P||Q) = Σ_(i≠j) p_ij ln(p_ij / q_ij), ∂KL/∂y_i = 4 Σ_j (p_ij - q_ij)(y_i - y_j)(1 + |y_i - y_j|^2)^-1
4. Optimiser. Gradient descent with momentum Δy(t) = μ Δy(t-1) - η g ∂KL/∂y and per-coordinate adaptive gains g (increased by 0.2 while the gradient keeps the same direction, multiplied by 0.8 when it flips). During early exaggeration every p_ij is multiplied by a factor (default 12) with momentum 0.5: attractive forces dominate, clusters form as tight, mobile blobs and can slide past each other. The KL shown is always computed with the true P.
How to read a map honestly. Distances between far-apart clusters, cluster sizes and axis directions carry little meaning; low perplexity invents fragments and even structure in pure noise; two runs with different seeds differ. What t-SNE does preserve is local neighbourhoods, which is why the page reports trustworthiness (T = 1 - 2/(nk(2n-3k-1)) Σ_i Σ_(j in U_i(k)) (r(i,j) - k), where U_i(k) are the false neighbours that enter the k-nearest list in the map and r is the rank in the original space), the fraction of true k-nearest neighbours that remain neighbours, and the 1-nearest-neighbour class agreement. PCA is the linear baseline: it keeps global variance and often wins on smooth, nearly linear data, but folds non-linear structure.
Checks run in the page. (1) The mean 2^H recomputed from the stored P_j|i equals the requested perplexity; (2) Σ p_ij = 1 and P is symmetric; (3) the analytic gradient matches central finite differences of the KL on a 14-point problem; (4) the PCA eigenvalues sum to the trace; (5) the trustworthiness of a random layout equals the analytic chance level; (6) KL falls and a fitted map is far above chance.
Simplifications. Synthetic, hand-designed datasets (not patient data and not a diagnostic tool), exact O(n^2) t-SNE rather than Barnes-Hut or FFT-accelerated variants, n up to 600, one random seed per run.