t-SNE & UMAP

Train a neural net to recognise handwritten digits and it will build, inside itself, a 512-dimensional vector for each image — a point in a space so vast the human mind cannot picture it. Yet somehow similar images end up near each other in that space. A "7" lives close to other "7"s; a "1" is far from an "8".

The challenge is seeing that structure. You can't plot 512 dimensions. But you can ask: is there a 2D map that preserves the neighbourhoods — that keeps points close in 2D whenever they were close in 512D?

That is exactly what t-SNE (t-Distributed Stochastic Neighbour Embedding, van der Maaten & Hinton 2008) and UMAP (Uniform Manifold Approximation and Projection, McInnes et al. 2018) do. They are not compression algorithms like Huffman coding; they are neighbourhood-preserving projections. Run them on the embedding layer of a classifier and you get a scatter plot where clusters reveal what the network actually learned.

The status: both algorithms are heuristic optimisation methods — there is no proof that the 2D layout they find is globally optimal, and the output depends on hyperparameters (perplexity for t-SNE, n_neighbors for UMAP). But in practice they are indispensable: every major ML paper that inspects latent representations uses one or both.

Below are 200 points living in a 10-dimensional space, organised into four clusters. Each cluster is a Gaussian blob of 50 points centred at a random position in 10D. The demo runs a simplified neighbourhood-preserving projection: it computes pairwise distances in 10D, then uses gradient descent to position each point in 2D so that close-in-10D neighbours stay close in 2D.

<p class="hint">{{hint}}</p>
<canvas id="canvas" width="480" height="320"></canvas>
<div class="btns">
  <button id="btn-project" type="button">{{btn_project}}</button>
  <button id="btn-regen" type="button" class="ghost">{{btn_regen}}</button>
</div>
<div class="info" id="info">{{info_initial}}</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .6rem; line-height: 1.5; }
canvas { display: block; width: 100%; max-width: 480px; height: auto;
         border-radius: 10px; background: #f4f6f9; border: 1px solid #d0d8e4; }
.btns { display: flex; gap: .5rem; margin-top: .7rem; flex-wrap: wrap; }
button { font: 600 14px system-ui; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.info { font-size: .88rem; color: #555; margin-top: .45rem; min-height: 1.3em; }

// Code not found

Press Project to run the optimisation. Watch the four clusters — distinguished by colour — gradually pull apart as the algorithm discovers that the four groups of points have very different neighbourhoods. Press Regenerate to start with a fresh random dataset and see that the cluster structure always emerges, regardless of the random seed. The axis labels are meaningless (there are no physical x/y dimensions in the original data); only the relative distances between points carry information.

Neither t-SNE nor UMAP solves a simple linear problem. Here is what makes them hard — and fast:

• t-SNE cost: the Kullback-Leibler divergence between pairwise similarity distributions in high-D and 2D. Naïve computation is $O(n^{2})$ per gradient step (n = number of points). With the Barnes-Hut tree trick it falls to $O(n \log n)$, but a corpus of 1 million points is still slow.
• UMAP cost: approximates topological structure via a weighted graph in high-D, then minimises a cross-entropy objective in 2D. The key improvement is that it builds a k-nearest-neighbour graph once ($O(n \log n)$ with approximate NN libraries), then optimises only the graph edges — making UMAP dramatically faster on large datasets.
• Non-convex landscape: both objectives are highly non-convex. Gradient descent finds a local minimum, not a global one. Run t-SNE twice with different random initialisations and you get different pictures — both valid, neither provably best.
• Hyperparameter sensitivity: t-SNE's perplexity controls how many neighbours each point "pays attention to" (typical values 5–50). UMAP's n_neighbors does the same. These choices shape the resulting clusters dramatically — a known pitfall for misinterpretation.
• What is preserved: t-SNE preserves local structure faithfully but distorts global distances (clusters may float arbitrarily far apart). UMAP better preserves global structure. Neither is a faithful distance-preserving embedding.

The result is that both methods are solved in practice (highly tuned open-source libraries run them daily on millions of points), but their theoretical properties are still an active research area — convergence guarantees, stability under perturbation, and what exactly "structure" they preserve are all open questions.

The need to "see" high-dimensional data appears everywhere modern machine learning is applied:

• Neural-net embeddings: after training a classifier, project the penultimate layer onto 2D to check whether the learned representations actually separate classes. If cat-images cluster with dog-images, the net is confused — you can see it immediately.
• Natural language processing: word2vec and transformer embeddings live in 300–1024 dimensions. UMAP plots routinely show that semantically similar words and sentences cluster — "king", "queen", "prince" form a tight group, while "car", "truck", "bus" form another.
• Single-cell genomics (scRNA-seq): each cell is described by expression levels of ~20 000 genes. t-SNE and UMAP let biologists identify cell types at a glance — a landmark application that made these algorithms famous outside ML.
• Drug discovery: molecular fingerprints are high-dimensional; projecting a chemical library into 2D lets chemists spot families of similar compounds.
• Anomaly detection: outlier points in 2D projections often correspond to corrupted data, mislabelled examples, or novel patterns worth investigating.

The connection to k-means clustering is direct: UMAP is often run before k-means to give a visual sanity-check, or after to confirm the clusters k-means found match genuine structure. And like dimensionality reduction in general, both methods sit at the heart of modern exploratory data analysis.

t-SNE and UMAP do something that sounds impossible: they take data living in thousands of dimensions and lay it out in a picture you can actually read. They are not magic — the 2D axes are meaningless, hyperparameters matter, and the cost function has many local minima. But when you see four tight clusters emerge from a noisy 10-dimensional cloud, you are watching a real mathematical property of the data come to light.

The next time you look at a scatter plot coloured by neural-net class and the colours separate cleanly, remember: that picture is the result of a gradient-descent optimisation preserving neighbourhood structure across a staggering difference in dimension. And it is one of the clearest windows we have into what a machine learning model has — or has not — learned.