Where t-SNE and UMAP Matter

Introduction

Train a neural net to recognise handwritten digits and it will build, inside itself, a 512-dimensional vector for each image — a point in a space so vast the human mind cannot picture it. Yet somehow similar images end up near each other in that space. A "7" lives close to other "7"s; a "1" is far from an "8".

The challenge is seeing that structure. You can't plot 512 dimensions. But you can ask: is there a 2D map that preserves the neighbourhoods — that keeps points close in 2D whenever they were close in 512D?

That is exactly what t-SNE (t-Distributed Stochastic Neighbour Embedding, van der Maaten & Hinton 2008) and UMAP (Uniform Manifold Approximation and Projection, McInnes et al. 2018) do. They are not compression algorithms like Huffman coding; they are neighbourhood-preserving projections. Run them on the embedding layer of a classifier and you get a scatter plot where clusters reveal what the network actually learned.

The status: both algorithms are heuristic optimisation methods — there is no proof that the 2D layout they find is globally optimal, and the output depends on hyperparameters (perplexity for t-SNE, n_neighbors for UMAP). But in practice they are indispensable: every major ML paper that inspects latent representations uses one or both.

Watch Clusters Separate

Below are 200 points living in a 10-dimensional space, organised into four clusters. Each cluster is a Gaussian blob of 50 points centred at a random position in 10D. The demo runs a simplified neighbourhood-preserving projection: it computes pairwise distances in 10D, then uses gradient descent to position each point in 2D so that close-in-10D neighbours stay close in 2D.

<p class="hint">200 points in 10D (4 clusters of 50). Press <b>Project</b> to run gradient descent — close pairs attract, distant pairs repel — and watch the clusters separate in 2D.</p>
<canvas id="canvas" width="480" height="320"></canvas>
<div class="btns">
  <button id="btn-project" type="button">Project</button>
  <button id="btn-regen" type="button" class="ghost">Regenerate</button>
</div>
<div class="info" id="info">Click Project to start.</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .6rem; line-height: 1.5; }
canvas { display: block; width: 100%; max-width: 480px; height: auto;
         border-radius: 10px; background: #f4f6f9; border: 1px solid #d0d8e4; }
.btns { display: flex; gap: .5rem; margin-top: .7rem; flex-wrap: wrap; }
button { font: 600 14px system-ui; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.info { font-size: .88rem; color: #555; margin-top: .45rem; min-height: 1.3em; }

const fmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// Neighbourhood-preserving 2D projection demo (t-SNE / UMAP concept):
// 1. Generate 200 points in 10D with 4 Gaussian clusters.
// 2. Precompute pairwise 10D distances (spring rest lengths).
// 3. Initialise 2D positions randomly.
// 4. Spring-force gradient descent: each pair has a rest length proportional
// to its 10D distance. Pairs close in 10D get short springs (attract);
// pairs far in 10D get long springs (repel). After convergence the 4 clusters of 50 points pull apart visibly — this is the key insight behind t-SNE/UMAP.
const N = 200, DIM = 10, K = 4, MAX_ITER = 500, LR = 0.01, BATCH = 10;
const COLORS = ['#e63946', '#2a9d8f', '#e9c46a', '#457b9d'];

let pts10 = [], labels = [], pts2 = [], dist10 = null, median10 = 1;
let animId = null, iter = 0;

// Seeded PRNG (LCG)
let seed = Date.now();
function rand() {
  seed = (seed * 1664525 + 1013904223) & 0xffffffff;
  return ((seed >>> 0) / 0x100000000);
}
function randn() {
  let u = 0, v = 0;
  while (u === 0) u = rand();
  while (v === 0) v = rand();
  return Math.sqrt(-2 * Math.log(u)) * Math.cos(2 * Math.PI * v);
}

function generate() {
  seed = (Date.now() ^ (Math.random() * 0x100000000)) | 0;
  pts10 = []; labels = [];
  // 4 random cluster centres in 10D, well separated
  const centres = Array.from({ length: K }, () =>
    Array.from({ length: DIM }, () => randn() * 3)
  );
  for (let k = 0; k < K; k++) {
    for (let i = 0; i < N / K; i++) {
      pts10.push(centres[k].map(c => c + randn() * 0.6));
      labels.push(k);
    }
  }
  // Precompute pairwise 10D Euclidean distances
  dist10 = new Float32Array(N * N);
  const allDists = [];
  for (let i = 0; i < N; i++) {
    for (let j = i + 1; j < N; j++) {
      let d = 0;
      for (let d_ = 0; d_ < DIM; d_++) d += (pts10[i][d_] - pts10[j][d_]) ** 2;
      d = Math.sqrt(d);
      dist10[i * N + j] = d;
      dist10[j * N + i] = d;
      allDists.push(d);
    }
  }
  // Median 10D distance used to scale 2D spring rest lengths
  allDists.sort((a, b) => a - b);
  median10 = allDists[allDists.length >> 1];
  // Random 2D init
  pts2 = Array.from({ length: N }, () => [randn() * 0.5, randn() * 0.5]);
}

function step(steps) {
  // Spring-force gradient: for a random sample of pairs,
  // rest length in 2D = (10D distance / median10) * scale.
  // The spring pulls/pushes the 2D positions to match 10D proximity.
  const scale = 0.8;
  for (let s = 0; s < steps; s++) {
    const grad = Array.from({ length: N }, () => [0, 0]);
    for (let p = 0; p < N * 4; p++) {
      const i = (rand() * N) | 0;
      const j = (rand() * N) | 0;
      if (i === j) continue;
      const target = dist10[i * N + j] / median10 * scale;
      const dx = pts2[i][0] - pts2[j][0];
      const dy = pts2[i][1] - pts2[j][1];
      const d2 = Math.sqrt(dx * dx + dy * dy) + 1e-6;
      const stretch = d2 - target;
      const fx = stretch * dx / d2;
      const fy = stretch * dy / d2;
      grad[i][0] += fx; grad[i][1] += fy;
      grad[j][0] -= fx; grad[j][1] -= fy;
    }
    for (let i = 0; i < N; i++) {
      pts2[i][0] -= LR * grad[i][0];
      pts2[i][1] -= LR * grad[i][1];
    }
  }
}

function draw() {
  const canvas = document.getElementById('canvas');
  const ctx = canvas.getContext('2d');
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  // Compute bounding box with padding
  let minX = Infinity, maxX = -Infinity, minY = Infinity, maxY = -Infinity;
  for (const [x, y] of pts2) {
    minX = Math.min(minX, x); maxX = Math.max(maxX, x);
    minY = Math.min(minY, y); maxY = Math.max(maxY, y);
  }
  const pad = 28, W = canvas.width - 2 * pad, H = canvas.height - 2 * pad;
  const rangeX = maxX - minX + 1e-9, rangeY = maxY - minY + 1e-9;
  const scale = Math.min(W / rangeX, H / rangeY);
  const offX = pad + (W - rangeX * scale) / 2;
  const offY = pad + (H - rangeY * scale) / 2;
  // Draw points
  for (let i = 0; i < N; i++) {
    const cx = offX + (pts2[i][0] - minX) * scale;
    const cy = offY + (pts2[i][1] - minY) * scale;
    ctx.beginPath();
    ctx.arc(cx, cy, 4, 0, Math.PI * 2);
    ctx.fillStyle = COLORS[labels[i]];
    ctx.globalAlpha = 0.85;
    ctx.fill();
  }
  ctx.globalAlpha = 1;
  // Legend (top-left)
  ctx.font = '11px system-ui, sans-serif';
  for (let k = 0; k < K; k++) {
    ctx.fillStyle = COLORS[k];
    ctx.beginPath(); ctx.arc(16, 16 + k * 18, 5, 0, Math.PI * 2); ctx.fill();
    ctx.fillStyle = '#333';
    ctx.fillText(fmt('Cluster %k%%v%', { k: k, v: 1 }), 26, 20 + k * 18);
  }
}

function project() {
  if (animId) { cancelAnimationFrame(animId); animId = null; }
  iter = 0;
  const infoEl = document.getElementById('info');
  function frame() {
    step(BATCH);
    iter += BATCH;
    draw();
    infoEl.textContent = fmt('Step %iter% / %MAX_ITER%%v%', { iter: iter, MAX_ITER: MAX_ITER, v: iter >= MAX_ITER ? ' — done. Clusters are now separated in 2D.' : ' — optimising…' });
    if (iter < MAX_ITER) animId = requestAnimationFrame(frame);
    else animId = null;
  }
  frame();
}

document.getElementById('btn-project').addEventListener('click', project);
document.getElementById('btn-regen').addEventListener('click', () => {
  if (animId) { cancelAnimationFrame(animId); animId = null; }
  generate();
  draw();
  document.getElementById('info').textContent = 'New dataset generated. Click Project to run.';
  iter = 0;
});

// Initialise
generate();
draw();
document.getElementById('info').textContent = 'Dataset ready (10D, 4 clusters). Click Project to run the projection.';

<p class="hint">200 points in 10D (4 clusters of 50). Press <b>Project</b> to run gradient descent — close pairs attract, distant pairs repel — and watch the clusters separate in 2D.</p>
<canvas id="canvas" width="480" height="320"></canvas>
<div class="btns">
  <button id="btn-project" type="button">Project</button>
  <button id="btn-regen" type="button" class="ghost">Regenerate</button>
</div>
<div class="info" id="info">Click Project to start.</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .6rem; line-height: 1.5; }
canvas { display: block; width: 100%; max-width: 480px; height: auto;
         border-radius: 10px; background: #f4f6f9; border: 1px solid #d0d8e4; }
.btns { display: flex; gap: .5rem; margin-top: .7rem; flex-wrap: wrap; }
button { font: 600 14px system-ui; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.info { font-size: .88rem; color: #555; margin-top: .45rem; min-height: 1.3em; }

const fmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// Neighbourhood-preserving 2D projection demo (t-SNE / UMAP concept):
// 1. Generate 200 points in 10D with 4 Gaussian clusters.
// 2. Precompute pairwise 10D distances (spring rest lengths).
// 3. Initialise 2D positions randomly.
// 4. Spring-force gradient descent: each pair has a rest length proportional
// to its 10D distance. Pairs close in 10D get short springs (attract);
// pairs far in 10D get long springs (repel). After convergence the 4 clusters of 50 points pull apart visibly — this is the key insight behind t-SNE/UMAP.
const N = 200, DIM = 10, K = 4, MAX_ITER = 500, LR = 0.01, BATCH = 10;
const COLORS = ['#e63946', '#2a9d8f', '#e9c46a', '#457b9d'];

let pts10 = [], labels = [], pts2 = [], dist10 = null, median10 = 1;
let animId = null, iter = 0;

// Seeded PRNG (LCG)
let seed = Date.now();
function rand() {
  seed = (seed * 1664525 + 1013904223) & 0xffffffff;
  return ((seed >>> 0) / 0x100000000);
}
function randn() {
  let u = 0, v = 0;
  while (u === 0) u = rand();
  while (v === 0) v = rand();
  return Math.sqrt(-2 * Math.log(u)) * Math.cos(2 * Math.PI * v);
}

function generate() {
  seed = (Date.now() ^ (Math.random() * 0x100000000)) | 0;
  pts10 = []; labels = [];
  // 4 random cluster centres in 10D, well separated
  const centres = Array.from({ length: K }, () =>
    Array.from({ length: DIM }, () => randn() * 3)
  );
  for (let k = 0; k < K; k++) {
    for (let i = 0; i < N / K; i++) {
      pts10.push(centres[k].map(c => c + randn() * 0.6));
      labels.push(k);
    }
  }
  // Precompute pairwise 10D Euclidean distances
  dist10 = new Float32Array(N * N);
  const allDists = [];
  for (let i = 0; i < N; i++) {
    for (let j = i + 1; j < N; j++) {
      let d = 0;
      for (let d_ = 0; d_ < DIM; d_++) d += (pts10[i][d_] - pts10[j][d_]) ** 2;
      d = Math.sqrt(d);
      dist10[i * N + j] = d;
      dist10[j * N + i] = d;
      allDists.push(d);
    }
  }
  // Median 10D distance used to scale 2D spring rest lengths
  allDists.sort((a, b) => a - b);
  median10 = allDists[allDists.length >> 1];
  // Random 2D init
  pts2 = Array.from({ length: N }, () => [randn() * 0.5, randn() * 0.5]);
}

function step(steps) {
  // Spring-force gradient: for a random sample of pairs,
  // rest length in 2D = (10D distance / median10) * scale.
  // The spring pulls/pushes the 2D positions to match 10D proximity.
  const scale = 0.8;
  for (let s = 0; s < steps; s++) {
    const grad = Array.from({ length: N }, () => [0, 0]);
    for (let p = 0; p < N * 4; p++) {
      const i = (rand() * N) | 0;
      const j = (rand() * N) | 0;
      if (i === j) continue;
      const target = dist10[i * N + j] / median10 * scale;
      const dx = pts2[i][0] - pts2[j][0];
      const dy = pts2[i][1] - pts2[j][1];
      const d2 = Math.sqrt(dx * dx + dy * dy) + 1e-6;
      const stretch = d2 - target;
      const fx = stretch * dx / d2;
      const fy = stretch * dy / d2;
      grad[i][0] += fx; grad[i][1] += fy;
      grad[j][0] -= fx; grad[j][1] -= fy;
    }
    for (let i = 0; i < N; i++) {
      pts2[i][0] -= LR * grad[i][0];
      pts2[i][1] -= LR * grad[i][1];
    }
  }
}

function draw() {
  const canvas = document.getElementById('canvas');
  const ctx = canvas.getContext('2d');
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  // Compute bounding box with padding
  let minX = Infinity, maxX = -Infinity, minY = Infinity, maxY = -Infinity;
  for (const [x, y] of pts2) {
    minX = Math.min(minX, x); maxX = Math.max(maxX, x);
    minY = Math.min(minY, y); maxY = Math.max(maxY, y);
  }
  const pad = 28, W = canvas.width - 2 * pad, H = canvas.height - 2 * pad;
  const rangeX = maxX - minX + 1e-9, rangeY = maxY - minY + 1e-9;
  const scale = Math.min(W / rangeX, H / rangeY);
  const offX = pad + (W - rangeX * scale) / 2;
  const offY = pad + (H - rangeY * scale) / 2;
  // Draw points
  for (let i = 0; i < N; i++) {
    const cx = offX + (pts2[i][0] - minX) * scale;
    const cy = offY + (pts2[i][1] - minY) * scale;
    ctx.beginPath();
    ctx.arc(cx, cy, 4, 0, Math.PI * 2);
    ctx.fillStyle = COLORS[labels[i]];
    ctx.globalAlpha = 0.85;
    ctx.fill();
  }
  ctx.globalAlpha = 1;
  // Legend (top-left)
  ctx.font = '11px system-ui, sans-serif';
  for (let k = 0; k < K; k++) {
    ctx.fillStyle = COLORS[k];
    ctx.beginPath(); ctx.arc(16, 16 + k * 18, 5, 0, Math.PI * 2); ctx.fill();
    ctx.fillStyle = '#333';
    ctx.fillText(fmt('Cluster %k%%v%', { k: k, v: 1 }), 26, 20 + k * 18);
  }
}

function project() {
  if (animId) { cancelAnimationFrame(animId); animId = null; }
  iter = 0;
  const infoEl = document.getElementById('info');
  function frame() {
    step(BATCH);
    iter += BATCH;
    draw();
    infoEl.textContent = fmt('Step %iter% / %MAX_ITER%%v%', { iter: iter, MAX_ITER: MAX_ITER, v: iter >= MAX_ITER ? ' — done. Clusters are now separated in 2D.' : ' — optimising…' });
    if (iter < MAX_ITER) animId = requestAnimationFrame(frame);
    else animId = null;
  }
  frame();
}

document.getElementById('btn-project').addEventListener('click', project);
document.getElementById('btn-regen').addEventListener('click', () => {
  if (animId) { cancelAnimationFrame(animId); animId = null; }
  generate();
  draw();
  document.getElementById('info').textContent = 'New dataset generated. Click Project to run.';
  iter = 0;
});

// Initialise
generate();
draw();
document.getElementById('info').textContent = 'Dataset ready (10D, 4 clusters). Click Project to run the projection.';

Result

Console

Press Project to run the optimisation. Watch the four clusters — distinguished by colour — gradually pull apart as the algorithm discovers that the four groups of points have very different neighbourhoods. Press Regenerate to start with a fresh random dataset and see that the cluster structure always emerges, regardless of the random seed. The axis labels are meaningless (there are no physical x/y dimensions in the original data); only the relative distances between points carry information.

The Real Complexity

Neither t-SNE nor UMAP solves a simple linear problem. Here is what makes them hard — and fast:

t-SNE cost: the Kullback-Leibler divergence between pairwise similarity distributions in high-D and 2D. Naïve computation is $O(n^{2})$ per gradient step (n = number of points). With the Barnes-Hut tree trick it falls to $O(n \log n)$ , but a corpus of 1 million points is still slow.
UMAP cost: approximates topological structure via a weighted graph in high-D, then minimises a cross-entropy objective in 2D. The key improvement is that it builds a k-nearest-neighbour graph once ( $O(n \log n)$ with approximate NN libraries), then optimises only the graph edges — making UMAP dramatically faster on large datasets.
Non-convex landscape: both objectives are highly non-convex. Gradient descent finds a local minimum, not a global one. Run t-SNE twice with different random initialisations and you get different pictures — both valid, neither provably best.
Hyperparameter sensitivity: t-SNE's perplexity controls how many neighbours each point "pays attention to" (typical values 5–50). UMAP's n_neighbors does the same. These choices shape the resulting clusters dramatically — a known pitfall for misinterpretation.
What is preserved: t-SNE preserves local structure faithfully but distorts global distances (clusters may float arbitrarily far apart). UMAP better preserves global structure. Neither is a faithful distance-preserving embedding.

The result is that both methods are solved in practice (highly tuned open-source libraries run them daily on millions of points), but their theoretical properties are still an active research area — convergence guarantees, stability under perturbation, and what exactly "structure" they preserve are all open questions.

Where It Matters

The need to "see" high-dimensional data appears everywhere modern machine learning is applied:

Neural-net embeddings: after training a classifier, project the penultimate layer onto 2D to check whether the learned representations actually separate classes. If cat-images cluster with dog-images, the net is confused — you can see it immediately.
Natural language processing: word2vec and transformer embeddings live in 300–1024 dimensions. UMAP plots routinely show that semantically similar words and sentences cluster — "king", "queen", "prince" form a tight group, while "car", "truck", "bus" form another.
Single-cell genomics (scRNA-seq): each cell is described by expression levels of ~20 000 genes. t-SNE and UMAP let biologists identify cell types at a glance — a landmark application that made these algorithms famous outside ML.
Drug discovery: molecular fingerprints are high-dimensional; projecting a chemical library into 2D lets chemists spot families of similar compounds.
Anomaly detection: outlier points in 2D projections often correspond to corrupted data, mislabelled examples, or novel patterns worth investigating.

The connection to k-means clustering is direct: UMAP is often run before k-means to give a visual sanity-check, or after to confirm the clusters k-means found match genuine structure. And like dimensionality reduction in general, both methods sit at the heart of modern exploratory data analysis.

Conclusion

t-SNE and UMAP do something that sounds impossible: they take data living in thousands of dimensions and lay it out in a picture you can actually read. They are not magic — the 2D axes are meaningless, hyperparameters matter, and the cost function has many local minima. But when you see four tight clusters emerge from a noisy 10-dimensional cloud, you are watching a real mathematical property of the data come to light.

The next time you look at a scatter plot coloured by neural-net class and the colours separate cleanly, remember: that picture is the result of a gradient-descent optimisation preserving neighbourhood structure across a staggering difference in dimension. And it is one of the clearest windows we have into what a machine learning model has — or has not — learned.

Comments

Loading comments...

https://www.kipuhub.com/en/article/t-sne-umap/Content licensed under CC BY-NC 4.0.

Cookies & privacy

t-SNE & UMAP

The standard lens for seeing what a neural net learned