Recap and final exam

Ten modules to go from the two ceilings of recurrent networks to a hand-built Transformer that translates dates and shows readable attention maps. Here is the course condensed, then the threads running through it.

The course at a glance

Module	What to retain
1. Recurrent limits	Sequentiality and the fixed-vector bottleneck are structural, not tuning problems
2. Query, key, value	Attention is a weighted average of values with softmax weights of scaled dot products
3. Multi-head attention	Several heads at $d_k = d / h$; total parameters stay $4 d^2$; heads specialise but many are redundant
4. Positional encoding	Sinusoidal, learned, or rotary: without one, attention is order-blind
5. Residuals and LayerNorm	Pre-norm and residuals are what make deep stacks trainable; FFN dominates parameter count
6. Encoder and BERT	Bidirectional attention + MLM; [CLS] for classification, tokens for extraction
7. Decoder and GPT	Causal mask + next-token prediction; KV cache is the memory hog at inference
8. Encoder-decoder and T5	Cross-attention connects the two halves; "everything is text-to-text"
9. Quadratic cost	Attention is $\mathcal{O}(n^2)$; FlashAttention, windows and sparse patterns push the ceiling
10. Implementing the block	Two hundred lines of PyTorch, trained on dates, attention maps as proof of correctness

The threads running through the course

Attention removes recurrence, not cost. The Transformer parallelises training beautifully — every position at once, one matmul per layer — but it pays an $\mathcal{O}(n^2)$ price on sequence length that recurrent networks did not have. The whole industry is now working on ways to keep the parallelism while flattening the quadratic curve, and the four families in module 9 are the current answer. Whenever you see a headline about a "million-token context", ask where the quadratic went.

One skeleton, three families. Modules 6 to 8 all built on the same layer from module 5. The difference between BERT, GPT and T5 is not the block; it is the mask and the objective. Bidirectional attention plus MLM gives an understanding model. Causal mask plus next-token prediction gives a generator. Both halves connected by cross-attention give a conditional generator. The choice of family is the biggest lever you have at model-selection time — scale is the second.

The mask is structural, not decorative. The most common bug in a hand-built decoder is a forgotten or misplaced causal mask. Training loss drops to zero (because the model peeks at the next token), and inference produces nonsense. The mask has to be a property of every attention layer in the decoder, and every implementation asserts its presence. If your generation looks great during training and terrible afterwards, the mask is the first suspect.

Attention maps are debugging tools, not just decorations. On any task where the correct answer is visible in the input at specific positions — copy, reverse, date translation, extractive QA — the cross-attention weights should light up on those positions. A trained Transformer that does not produce clean attention maps on such tasks is misconfigured, whatever its loss says. The date-translation notebook of module 10 is the reference: three bright regions, aligned with year, month and day.

The final exam

The exam has 40 questions covering all ten modules: computing attention by hand on three tokens (module 2), sizing the head dimension given a target $d$ and $h$ (module 3), picking a positional scheme for a task with long extrapolation (module 4), diagnosing a training loss that plateaus (module 5), choosing between BERT, GPT and T5 for a given task (modules 6 to 8), catching a missing causal mask from its symptoms (module 7), estimating attention memory as $n$ grows (module 9), and reading unit tests on Transformer shapes (module 10).

Several questions present situations to diagnose: a model that trains to zero loss then generates nonsense, an encoder that scores worse than expected on a classification task because a batch normalisation slipped in, a validation loss that regresses after switching from pre-norm to post-norm without warmup, an attention map that stays diffuse for every input. Judgement is what gets assessed, not memorised paper equations.

On success, your certificate of completion is issued immediately; its number can be verified by any third party on the platform.

Before you start

Go back over the table above and, for each row, ask yourself "what symptom would I see if I got this wrong?". If you can explain why the scaling by $\sqrt{d_k}$ matters, why removing the causal mask is invisible during training, and why a "long context" model can still miss information placed at position 60k, you are ready. Good luck!

Final exam

Ready to validate this course?

40 questions drawn at random from the course bank · passing score 70% · verifiable PDF certificate issued immediately on success.

Start the exam

You need to be signed in to your InSkillML account with an active subscription. You can also start the exam from My courses.