📄️ RNN and LSTM

Recurrent neural networks from the ground up: hidden state, LSTM and GRU gates, encoder-decoder for translation. Duration 6h, applied project, 40-question exam and verifiable certificate.

📄️ 1. Sequence vs table

Module 1 of the RNN and LSTM premium course: why the order of observations is a signal, how sliding windows turn a series into supervised examples, and where temporal leakage hides in a naive split.

📄️ 2. Recurrent neuron

Module 2 of the RNN and LSTM premium course: recurrence equation, unrolling in time, weight sharing, return_sequences, and how a (batch, time, features) tensor flows through a recurrent layer.

📄️ 3. BPTT

Module 3 of the RNN and LSTM premium course: unrolled gradient, memory cost proportional to sequence length, truncated BPTT, and why long sequences quickly stop fitting in a laptop's memory.

Module 4 of the RNN and LSTM premium course: the product of Jacobians, a 50-step numerical demonstration, gradient clipping, orthogonal initialisation, and why a SimpleRNN cannot learn long dependencies.

📄️ 5. LSTM

Module 5 of the RNN and LSTM premium course: cell state versus hidden state, the three gates, why gradient flows through the additive path, and how to count parameters of an LSTM layer.

📄️ 6. GRU

Module 6 of the RNN and LSTM premium course: update and reset gates, LSTM versus GRU parameter counts and results on the electricity red thread, and when the simpler cell is the better default.

📄️ 7. Bi and stacked

Module 7 of the RNN and LSTM premium course: when the future is available and when it is not, stacking layers, recurrent dropout, and the mistake of using a bidirectional model for forecasting.

📄️ 8. Encoder-decoder

Module 8 of the RNN and LSTM premium course: context vector, teacher forcing, start and end tokens, greedy decoding, and the bottleneck that motivates attention in course 12.

📄️ 9. Padding and batching

Module 9 of the RNN and LSTM premium course: sliding windows, padding and masking, sorting by length, per-window normalisation without leakage, and building the pipeline with tf.data or DataLoader.

📄️ 10. Project

Module 10 of the RNN and LSTM premium course: end-to-end forecasting pipeline on hourly electricity consumption, naive baseline, LSTM against GRU, quantile intervals and the mistakes to avoid.

📄️ 11. Recap and exam

Complete recap of the RNN and LSTM premium course: sequences, hidden state, BPTT, gradient collapse, LSTM, GRU, bidirectional and stacked networks, encoder-decoder, pipelines, and the 40-question exam.

RNN & LSTM