Skip to main content

RNN and LSTM: modeling sequences with recurrent networks

Recurrent networks from scratch: how a network with a memory reads a sequence, why long dependencies collapse without gates, and how LSTM and GRU salvage the idea.

Course duration: 6h​

What you will learn​

  • Recognise the situations where the order of observations carries the signal, and where a table of features destroys it
  • Read the recurrence equation of a simple RNN and follow the shape of its tensors along the time axis
  • Explain why backpropagation through time is expensive in memory and unstable on long sequences
  • Choose an LSTM or a GRU on evidence rather than habit, and stack or bidirect them when the task allows it
  • Build an encoder-decoder for a short translation task and understand the bottleneck that motivates attention
  • Package the full pipeline for a hourly electricity forecast, with a naive baseline and quantile prediction intervals

Prerequisites​

  • Deep learning fundamentals (course 07)
  • A framework, either TensorFlow and Keras (course 08) or PyTorch (course 09)

The red thread​

Two datasets carry the ten modules. The first is the hourly electricity consumption of a small office building: a numeric series with clear daily and weekly cycles, sampled every hour, used for a 24-step-ahead forecasting problem. It appears in modules 1 to 7 and 9 to 10, so every architectural decision — hidden state size, LSTM versus GRU, stacked or bidirectional, padding strategy — can be judged on the same measurable task.

The second dataset is a small corpus of short French-English sentence pairs taken from a public sentence collection, kept to a few thousand pairs so the encoder-decoder in module 8 trains on a laptop in minutes rather than hours. It is the only module that leaves numeric forecasting behind, because the encoder-decoder architecture requires two sequences of different lengths to make sense.

The final project (module 10) returns to the electricity series and asks the harder question that every real forecasting project ends up asking: not just a single prediction, but a prediction interval obtained from quantile losses, because a point estimate without uncertainty is dangerous in operations.

Course modules​

  1. What makes a sequence different from a table of features
  2. The recurrent neuron and its hidden state
  3. Backpropagation through time
  4. Vanishing and exploding gradients
  5. LSTM: forget, input and output gates
  6. GRU: an effective simplification
  7. Bidirectional and stacked networks
  8. Encoder-decoder for translation
  9. Splitting, padding and batching sequences in practice
  10. Project: forecasting a series of measurements

Assessment and certificate​

The course ends with a 40-question exam covering every module. On success, a certificate of completion is issued immediately; its number can be verified by anyone on the platform.

Free courses, by contrast, end with a 5-question quiz and a certificate preview, without certification.