Recap and final exam
Ten modules to go from a lone tensor to a Fashion-MNIST classifier served over HTTP. Here is the course condensed, then the threads running through it, then the exam.
The course at a glance
| Module | What to retain |
|---|---|
| 1. Tensors | Shape, dtype, device; from_numpy shares memory, tensor copies; in-place breaks autograd |
| 2. Autograd | Dynamic graph, gradients accumulate; no_grad and detach do different things |
3. nn.Module | Register via __setattr__; a plain list of layers is invisible to parameters() |
| 4. Dataset and DataLoader | Split first, compute statistics second; num_workers and pin_memory unblock the pipeline |
| 5. Training loop | Five ordered steps; CrossEntropyLoss wants logits; eval() and no_grad() are complementary |
| 6. Optimizers and schedulers | AdamW for modern classifiers; OneCycleLR steps per iteration, CosineAnnealingLR per epoch |
| 7. GPU and mixed precision | Overlap transfers with compute; GradScaler mandatory with float16 |
| 8. Checkpoints | Save five things, not one; map_location makes checkpoints portable; last epoch is not best epoch |
| 9. Transfer learning | Two phases in order; requires_grad=False does not freeze BatchNorm running statistics |
| 10. TorchScript and ONNX | trace freezes control flow, script compiles it; always verify numerical parity |
The threads running through the course
Two modes coexist and PyTorch will not warn you. Dropout is on during training and off during inference. BatchNorm computes batch statistics during training and uses running ones during inference. Both switches are controlled by model.train() and model.eval(), and both are silent when wrong. The evaluation phase of module 5 needs eval(). The frozen backbone of module 9 needs eval() on every BatchNorm. The exported model of module 10 needs eval() before tracing. Three different places, one mechanism, one recurring bug. Recognising it once means recognising it everywhere.
Gradients accumulate, and this shapes the entire loop. zero_grad is the first step of every iteration for the exact reason backward adds to .grad rather than replacing it. This design lets you sum gradients over sub-batches for gradient accumulation — a useful trick when memory is tight — but the default behaviour catches almost every beginner, exactly once. Making the five-step pattern reflexive — zero_grad, forward, loss, backward, step — is what prevents the accumulation trap and, at the same time, prepares you to break the pattern deliberately when a real reason emerges.
What is not in the artifact does not exist. A normalisation computed on the fly in the training script and forgotten at inference. A preprocessing pipeline that exists only in a notebook cell. A weights_only=False checkpoint that pickled a class that has since been renamed. In every case, information required to reproduce the training result lives outside the object that is supposed to contain everything. TorchScript and ONNX exist to solve this, but only when the export step captures the whole pipeline — inputs, outputs, transforms, eval() mode. Before deploying, the question to ask is always: does this artifact contain everything, or does it silently depend on my training environment?
The bottleneck is almost never where you look for it. The instinctive answer to slow training is more hardware. Modules 4, 7 and 9 propose the reverse order: measure with a profiler, fix the data pipeline first, enable mixed precision second, and only then look at model size or distribution. The first two cost nothing and often win more than the third, which multiplies the bill. Every performance decision in this course is preceded by "measure it".
The final exam
The exam has 40 questions drawn from a pool of 48, covering all ten modules: the difference between a view and a copy, why in-place breaks autograd, the accumulation of .grad and its remedy, the interplay of eval() and no_grad(), how a Dataset differs from a DataLoader, the five ordered steps of a training iteration, when to use AdamW rather than Adam, how OneCycleLR and CosineAnnealingLR differ in cadence, why GradScaler matters for float16 but not bfloat16, the five items a resume-capable checkpoint contains, the two-phase order of transfer learning, and the specific traps of torch.jit.trace versus torch.jit.script.
Several questions present situations to diagnose: a training loss that stops falling after a resume, a fine-tuning run whose accuracy regresses despite frozen weights, a served model that returns different predictions from the local one, a validation number that looks great and collapses in production. Judgment is what gets assessed, not memorised API signatures.
On success, your certificate of completion is issued immediately; its number can be verified by any third party on the platform.
Reread the table above and, for each row, ask yourself "what symptom would I see if I got this wrong?". If you can explain why a forgotten zero_grad degrades training silently, why requires_grad=False is not enough to freeze a BatchNorm, and why a traced model can return wrong answers on inputs that look correct, you are ready. Good luck!
Final exam
Ready to validate this course?
40 questions drawn at random from the course bank · passing score 70% · verifiable PDF certificate issued immediately on success.
Start the examYou need to be signed in to your InSkillML account with an active subscription. You can also start the exam from My courses.