Recap and final exam

Ten modules to move from a single neuron to a network trained and diagnosed. Here is the course condensed, then the threads that run through it.

The course at a glance

Module	The essential point
1. Perceptron to network	Weighted sum then activation; the perceptron fails on XOR, depth is efficient rather than necessary
2. Activation functions	Without non-linearity, a hundred layers equal one; ReLU by default, GELU for transformers
3. Forward pass	A layer is a matrix product; the loss must receive raw scores, not a softmax
4. Backpropagation	Chain rule: redistribute through $W^{\top}$, filter by $f'(z)$, in a single backward pass
5. Optimizers	The learning rate dominates everything; AdamW with warmup then cosine decay
6. Initialization and gradients	He for ReLU; the $+1$ of a residual connection opens a direct path for the gradient
7. Regularization	Dropout equals an ensemble of sub-networks; inverted dropout makes eval() mandatory at inference
8. Normalization	$\gamma$ and $\beta$ make the transformation optional; per layer as soon as you leave vision
9. Learning curves	A flat loss is a bug; overfitting ten observations validates the chain in two minutes
10. First network	Split before preprocessing; save the scaler with the weights

The threads running through the course

Everything comes back to the backpropagation formula. A deep weight's gradient is a product of local derivatives: that single sentence explains the vanishing gradient (module 6), why ReLU supplanted the sigmoid (module 2), why a saturated activation blocks learning, and why residual connections work. Understanding that formula gives you a mental model that makes the other modules deducible rather than memorizable.

Training and inference are two distinct regimes. Dropout disables neurons in one, not the other. Batch normalization uses batch statistics, then running averages. Forgetting eval() raises no error: it silently degrades predictions and makes them batch-dependent. It is this course's most frequent bug.

The loss and the output activation form a couple. Softmax with categorical cross-entropy, sigmoid with binary cross-entropy, linear output with squared error. This is not an arbitrary convention: it is that pairing which produces the $\delta^{(L)} = \hat{y} - y$ simplification of module 4.

Diagnosis precedes architecture. Facing a network that will not learn, the temptation is to change models. Modules 6 and 9 propose the opposite: measure gradient norms, read the two curves, overfit a small sample. Those three gestures locate the cause in minutes, where architectural guesswork costs hours.

The final exam

The exam has 40 questions covering the ten modules: neuron computation and layer sizing, choosing activation and loss by task, the mechanics of backpropagation, tuning the learning rate and choosing an optimizer, initialization and gradient diagnosis, dropout and normalization with their two regimes, reading learning curves, and building a correct training loop.

Several questions present situations to diagnose: a loss going NaN, curves diverging, a model whose predictions change with the batch, training stuck from the first epoch. It is judgment that is assessed, not the recitation of formulas.

On success, your certificate of completion is issued immediately; its number is verifiable by any third party on the platform.

Before you start

Take the table above and, for each row, ask yourself "how would I see that I am wrong here?". If you can say why a hundred layers without activation equal one, why a model gives different predictions depending on its batch, and what a flat loss from the first epoch means, you are ready. Good luck!

Final exam

Ready to validate this course?

40 questions drawn at random from the course bank · passing score 70% · verifiable PDF certificate issued immediately on success.

Start the exam

You need to be signed in to your InSkillML account with an active subscription. You can also start the exam from My courses.