Skip to main content

Reinforcement learning

An agent that acts, an environment that reacts, a reward that scores the last decision: from this loop, we build agents that solve FrozenLake with a table, CartPole with a small network, and LunarLander with proximal policy optimization.

Course Duration: 8h​

What You'll Learn​

  • Frame a control problem as a Markov decision process and know when the Markov assumption is violated
  • Compute value functions with dynamic programming when the model is known, with Monte Carlo when it is not
  • Implement tabular Q-learning and diagnose an exploration schedule that fails
  • Approximate a Q-function with a neural network, stabilize training with a replay buffer and a target network
  • Move from value-based to policy-based methods with REINFORCE, then variance reduction with a baseline
  • Combine actor and critic into A2C and PPO, and compare a homemade implementation against Stable-Baselines3
  • Read reward curves with seed variability in mind and know what an RL result actually proves
  • Recognize what does and does not transfer from a simulator to the real world

Prerequisites​

  • Mathematics for AI (course 03): probability, expected value, gradient
  • Deep learning fundamentals (course 07): a network, a loss, an optimizer
  • PyTorch (course 09): tensors, nn.Module, an optimizer step

Course Modules​

  1. Agent, environment, reward: the framework
  2. Markov decision processes
  3. Value functions and Bellman equations
  4. Dynamic programming and Monte Carlo methods
  5. Temporal difference and Q-learning
  6. Exploration: epsilon-greedy and alternatives
  7. Deep Q-networks and experience replay
  8. Policy gradient methods
  9. Actor-critic, A2C and PPO
  10. Project: an agent trained on a Gymnasium environment

The thread that runs through the course​

Three Gymnasium environments of increasing difficulty carry the same agent across every module: FrozenLake for the tabular chapters (2 to 6), CartPole for the first deep methods (DQN in module 7, REINFORCE in module 8), and LunarLander for actor-critic and the final project (PPO in modules 9 and 10). The reward-per-episode curve is plotted at every stage, so the same yardstick measures a Q-table and a neural policy. Libraries used: Gymnasium for environments, PyTorch for networks, and Stable-Baselines3 in module 9 as a reference against the homemade PPO.

Assessment and certificate​

The course ends with a 40-question exam covering every module. On success, a certificate of completion is issued; its number is verifiable on the platform.

Free courses, by contrast, end with a 5-question quiz and a preview of the certificate, without certification.