#t-sne-umap — Unsupervised learning

Map the high dimensions: t-SNE and UMAP unfold 10-dimensional data into a readable 2D map — perplexity, neighbors, and reading pitfalls.

What you'll play with

Welcome to #t-sne-umap. On screen, 100 points that live in 10 dimensions: four groups of 25, colored by their true label — a color the algorithm will never see. On the left, PCA flattens them into 2D: it keeps the direction of greatest variance, here a shared noise axis, and superimposes the groups. On the right, the t-SNE map at iteration 0: still the PCA, but t-SNE is going to unfold it by looking only at each point's neighborhoods.
Run the first 50 iterations: /iterate 50. This is the early exaggeration phase: affinities between neighbors are multiplied by 4, the map contracts and groups form at once.
Refinement: /iterate 100. Exaggeration stops, the map relaxes and the four clouds spread out clearly. Remember: the colors have never been shown to the algorithm, it has only seen 10D distances.
Perplexity is the effective number of neighbors each point takes into account. Make it tiny: /perplexity 5. Positions reset to zero (iteration 0).
Run /iterate 100 to see the effect of perplexity 5: lumps, stretched groups. Then swing to the opposite extreme: /perplexity 50 — half the points count as neighbors.
Run /iterate 100 if you want to see the perplexity-50 map: everything spreads out, the space between groups no longer means anything. Now change method: /method umap. UMAP starts from the same idea, but builds a k-nearest neighbors graph and then optimizes it with forces: attraction along edges, repulsion elsewhere.
Run /iterate 100 to let UMAP settle. Then hide the reference to enlarge the map: /compare.
Let's change the data shape: /dataset chain. This time the 100 points follow a curve in 10D, colored in four segments from start to end: the true structure is a line.
Your turn: /dataset mixed (three groups of 40, 20 and 10 points plus a chain — see how t-SNE gives them comparable sizes), /method tsne then /iterate 100, /neighbors 5 for a very local UMAP, /lr 500 to see the descent go wild, /seed 42 for another draw, /reset to start over. Going further: #pca for the linear version of this story, and #embeddings-3d for what these maps are used to look at.

Channel commands

/iterate <n=10..100> — Run n iterations of the current method (at most 600 in total).
/perplexity <5..50> — Set the t-SNE perplexity (effective number of neighbors) and reset positions.
/neighbors <5..30> — Set the number k of neighbors for the UMAP graph and reset positions.
/method <tsne|umap> — Choose t-SNE or UMAP and reset positions.
/lr <10..500> — Set the learning rate (without resetting).
/compare — Show or hide the PCA reference map.
/dataset <clusters|chain|mixed> — Change the dataset (100 points in 10D) and reset positions.
/seed <1..99> — Redraw the dataset with another seed and reset positions.
/reset — Return to the clusters dataset, t-SNE, perplexity 30, PCA shown.

Glossary

t-SNE: Non-linear dimensionality reduction that places each point in 2D so that its neighborhoods resemble those in the original space: Gaussian affinities in high dimensions, Student kernel in 2D, and gradient descent on the Kullback-Leibler divergence between the two.
UMAP: Non-linear dimensionality reduction that builds a weighted k-nearest neighbors graph (fuzzy graph), then optimizes a 2D map with forces: attraction along the edges, repulsion of negative samples. Faster than t-SNE and a bit more faithful to global structure.
perplexity: t-SNE parameter: the effective number of neighbors each point takes into account (2 to the power of the entropy of its affinities). The width σᵢ of each point's Gaussian is tuned to reach it. Usual values: 5 to 50.
non-linear dimensionality reduction: Representing high-dimensional data in 2 or 3 dimensions (an embedding) without limiting yourself to a projection: the map can fold, stretch or tear space to preserve what matters, often local neighborhoods.
Kullback-Leibler divergence: Measure of gap between two probability distributions, here the P affinities (high dimensions) and Q (2D map). t-SNE minimizes it; it heavily penalizes close 10D neighbors placed far apart in 2D, and lightly the opposite.
early exaggeration: t-SNE trick: multiply the P affinities (here by 4) during the first iterations. The map contracts, groups form fast and well separated, then exaggeration stops and the map relaxes.
heavy-tailed Student distribution: Kernel used by t-SNE in the 2D map: 1 / (1 + d²). Its heavier tail than a Gaussian lets non-neighbors drift far apart at no cost, which avoids central crowding and pushes groups apart.
k-nearest neighbors graph: A graph linking each point to its k nearest neighbors in the original space. UMAP weights it (weights exp(−(d − ρᵢ)/σᵢ)), symmetrizes it into a fuzzy graph, then works only with it: the map just has to respect these links.
neighborhood preservation: Fidelity measure of a map: share of each point's k nearest neighbors in high dimensions found among its k nearest in 2D, on average. 100% = all neighborhoods respected.
reading artifacts: What a t-SNE or UMAP map does not measure: the distance between two groups, the size or density of a group, and sometimes the very existence of groups (too small a perplexity fragments a continuous cloud). Only neighborhood composition is reliable.

Other channels in Unsupervised learning

#k-means — Grouping without labels: centroids that move, inertia that drops, the choice of k — and the shapes where k-means fails.
#pca — Principal component analysis: find the axes where data varies most, project, compress — and measure what is lost.
#hierarchical-clustering — Merge the points two at a time until only one is left: the dendrogram, the linkage criteria, and the cut height that decides the number of clusters.
#dbscan — Group by density: epsilon, MinPts, core, border and noise points — the algorithm that finds arbitrary shapes and ignores intruders.
#anomaly-detection — Spot what fits nothing: z-score / Mahalanobis, Isolation Forest, LOF — three ways to say 'this point is odd'.
#t-sne-umap — Map the high dimensions: t-SNE and UMAP unfold 10-dimensional data into a readable 2D map — perplexity, neighbors, and reading pitfalls.