Recap and final exam

Ten modules to go from "what does it mean to generate?" to a small diffusion model conditioned on text, then to the responsibilities that come with shipping any of it. Here is the course condensed, then the threads running through it, then the exam.

The course at a glance

Module	What to retain
1. Discriminative vs generative	Discriminative models $p(y \mid x)$, generative models $p(x)$; three families, three routes around intractable likelihood
2. Autoencoders	A bottleneck compresses; random codes decoded produce noise; latent space has empty regions
3. Variational autoencoders	Reparameterisation $z = \mu + \sigma \varepsilon$, KL against $\mathcal{N}(0, I)$; blurry because squared error averages
4. GAN	Two-player game with non-saturating loss; DCGAN recipe; loss curves are almost useless as a quality signal
5. Mode collapse	Symptoms and fixes: Wasserstein with gradient penalty, spectral normalisation, TTUR; diagnose with class distribution, not losses
6. Diffusion models	Forward noising is closed-form; predict noise with a U-Net conditioned on $t$; DDPM is exact, DDIM is fast
7. Text conditioning	CLIP embeddings, cross-attention, classifier-free guidance; the guidance scale is a real dial with bad values past $\sim 15$
8. FID and metrics	FID beats inception score by comparing distributions; both fail to detect memorisation; plan for human evaluation
9. Text, image, audio	Autoregressive wins on text, diffusion on images, audio is split by task; cost per sample varies 1000-fold
10. Copyright and provenance	Training data is legally unresolved; watermarking and C2PA are complementary; AI Act Article 50 is a reasonable baseline

Head-to-head: VAE, GAN, diffusion

Property	VAE	GAN	Diffusion
Sample quality	blurry	sharp	very sharp
Sample diversity	good	often narrow (mode collapse)	good
Sampling speed	very fast (one pass)	very fast (one pass)	slow (25 to 1000 steps)
Training stability	very stable	unstable, needs stabilisers	very stable
Likelihood	lower bound (ELBO)	undefined	variational bound (implicit)
Best for	interpolable latent space	latency-critical generation	quality, conditioning, control

The table is not a hierarchy: each family remains the right answer to a different question. Diffusion has won on unconditional and text-conditioned image quality, GANs remain the right answer for sub-millisecond latency, VAEs remain the right answer when a smooth interpretable latent space is the goal.

The threads running through the course

Sample quality and mode coverage are two different axes. A VAE covers the distribution but produces blurry samples. A GAN produces sharp samples but often misses whole classes. Diffusion is the first family to do both at once, at the cost of slow sampling. No single scalar metric captures both axes reliably — FID conflates them, inception score ignores the real data, and precision-recall decouples them. This is the reason human evaluation still matters in 2026.

Stability comes from the objective, not from tricks. The reason GANs are hard to train is not that practitioners lacked cleverness for a decade; it is that the minimax objective has no monotonic quality signal. The reason diffusion training is stable is not that engineers found the right learning rate; it is that the objective — a squared error against a known noise target — is a plain regression. When the objective is right, ordinary SGD is enough. When the objective is fundamentally hard, no combination of tricks fully rescues it.

The guidance scale is a trade-off, not a fidelity dial. Text conditioning ships with a knob called guidance scale, and users push it too far. Guidance is not a "listen more carefully" setting: it is a compromise between staying inside the natural image distribution and following the prompt. The knob has a good regime around 5 to 10, an over-saturated one past 15, and a broken one past 30. This is the most common practical mistake in prompt-driven generation.

Every generative system operates under a regulatory regime, and building for it costs less than retrofitting it. Provenance, consent testing, deepfake safeguards — treating them as compliance to add at the end is a common mistake and a costly one. The developer who bakes them into the training and evaluation pipeline from the first internal build pays a small fixed cost. The one who does not pays a much larger variable cost at the first incident.

The final exam

The exam has 40 questions covering all ten modules: distinguishing discriminative from generative modelling, the reparameterisation trick, the two terms of the VAE loss and why samples blur, the non-saturating GAN loss and why DCGAN uses betas=(0.5, 0.999), symptoms of mode collapse and which fix targets which symptom, the forward and reverse diffusion processes, why the U-Net is conditioned on the timestep, the difference between DDPM and DDIM sampling, cross-attention with CLIP embeddings, the role and pitfalls of classifier-free guidance and its scale, FID and its known limitations, precision-recall for generation, the modality-by-modality choice of family, the cost of generation and its non-linear scaling, memorisation, watermarking, C2PA and the outline of Article 50 of the AI Act.

Several questions present situations to diagnose: a VAE that produces sharp images but no diversity, a GAN whose losses look healthy but samples only produce three digit classes, a diffusion model that outputs nonsense when the timestep is passed as a float, a generator with excellent FID that turns out to memorise the training set, a text-to-image system that produces oversaturated images when the guidance scale is pushed to 25, a face generator that has never been tested against a specific celebrity's name. Judgement is what gets assessed, not memorised API signatures.

On success, your certificate of completion is issued immediately; its number can be verified by any third party on the platform.

Before you start

Go back over the head-to-head table above and, for each row, ask yourself "in what situation would I choose this family?". If you can explain why a VAE has a lower bound on the likelihood but blurs, why a GAN's losses are a bad quality signal, why diffusion needs the timestep as input, and what the guidance scale actually is, you are ready. Good luck!

Final exam

Ready to validate this course?

40 questions drawn at random from the course bank · passing score 70% · verifiable PDF certificate issued immediately on success.

Start the exam

You need to be signed in to your InSkillML account with an active subscription. You can also start the exam from My courses.