Lesson 1 — What generating means
A model that tells cats from dogs needs to find one thing: the boundary between them. Everything on one side is a cat, everything on the other is a dog, and the model can be entirely ignorant about what cats look like as long as it places the line correctly.
A model that draws a cat cannot get away with that. It needs to know that cats have four legs and two ears, that fur has a particular texture, that eyes reflect light, that legs attach to bodies in specific ways. It needs a model of what cats are like, not merely where they differ from dogs.
That is the whole distinction, and it explains the timeline.
Learning a boundary versus learning a distribution
Discriminative models learn the probability of a label given the data: what is this? They are the classifiers, detectors and predictors that made up most of applied machine learning for two decades. They need only enough information to separate the categories in front of them.
Generative models learn what the data itself looks like — the distribution from which real examples are drawn. Once you have that, you can sample from it, and a sample is a new example that was never in the training set.
| Discriminative | Generative | |
|---|---|---|
| Learns | Where categories separate | What the data looks like |
| Question answered | "What is this?" | "What could this be?" |
| Output | A label or a number | New content |
| Needs to know | Only the differences | The structure of the whole |
| Difficulty | Lower | Substantially higher |
The asymmetry is worth appreciating. You can distinguish two composers reliably without being able to compose in either style. Producing convincing work in one style requires knowing vastly more.
Why it took so long
Three obstacles, all of which had to be cleared.
The output space is enormous. A classifier picks one of a thousand labels. An image generator picks values for three million numbers, and almost every possible combination is noise. Valid images occupy a vanishingly small region of that space, and the model has to learn where it is.
Evaluation is genuinely hard. A classifier is right or wrong. A generated image is good or bad, and nobody can write that down as a differentiable objective. The absence of a clean training signal held the field back for years.
Coherence must hold globally. An image where each patch is individually plausible but the arm connects to the wrong shoulder fails completely. Local correctness is not enough, and enforcing global structure is difficult.
The succession of approaches
Variational autoencoders compress images into a small vector and reconstruct them. Since the compressed space is continuous, you can pick a point in it and decode to get a new image. The mechanism was elegant and the results were blurry, because the training objective rewarded averaging over committing to detail.
Generative adversarial networks solved the evaluation problem by inventing a critic. Train two networks: a generator producing images and a discriminator judging real from fake. Each improves against the other, and the discriminator supplies the training signal nobody could write by hand. GANs produced the first genuinely convincing generated faces, and they were notoriously unstable to train — the two networks could collapse into stalemate or the generator could learn to produce a single good image and stop.
Diffusion models superseded both by reframing the problem into something with a clean training signal, which is lesson 2.
Transformers did for text what diffusion did for images, using a different route: predict the next token, repeatedly. Lesson 3.
Latent space, and why it is useful
Most generative models operate in a latent space: a compressed representation with far fewer dimensions than the raw output. A 512 × 512 image has 786,432 numbers; its latent representation might have 16,000.
Two reasons this matters.
Computation. Working in the compressed space is dramatically cheaper, which is precisely why image generation runs on consumer hardware rather than requiring a data centre.
Structure. The latent space is organised meaningfully. Nearby points decode to similar images, and directions in the space correspond to interpretable changes — moving along one axis might add glasses to a face, another might change the season in a landscape. This is what makes controlled editing possible rather than only random sampling.
A generative model does not store examples and retrieve them. It stores a compressed, statistical account of what its training data was like, and sampling from that account produces something new. This is why the output is usually novel, why it can nonetheless echo work it saw many times, and why asking "which image did it copy?" is usually the wrong question — though not always, as lesson 5 covers.
What is generated today
| Domain | Maturity in 2026 |
|---|---|
| Text | Fluent and useful, unreliable on facts |
| Images | Extremely strong, occasional structural errors |
| Code | Very useful with review, confidently wrong at times |
| Audio and speech | Convincing, including voice cloning from seconds of sample |
| Music | Good for background and stock use |
| Video | Improving fast, short clips, temporal consistency still limited |
| 3D models | Early, usable as a starting point |
| Molecules and proteins | Genuinely productive in research |
The last row is worth noticing, since it rarely features in public discussion. Generative models proposing candidate protein structures and drug molecules have produced results that matter scientifically, and the value there comes from generating many candidates cheaply for expensive validation to filter — a pattern that generalises further than most people apply it.
In three sentences
A discriminative model learns only where categories separate, while a generative model learns what the data itself looks like, which is a much larger thing to know and the reason generation arrived two decades after reliable classification. The obstacles were an enormous output space where almost every combination is noise, the absence of any writable objective for "this looks good", and the need for global rather than local coherence — cleared in succession by autoencoders, GANs, diffusion and transformers. Most generative models work in a compressed latent space, which makes generation affordable and gives meaningful structure to sample and edit within, and none of them stores examples to retrieve: they store a statistical account of what their training data was like.