SS24: Introduction to Reinforcement Learning

This course aims at providing an introduction to reinforcement learning (RL) by covering the main topics ranging from classic approaches to modern techniques, with a particular focus on the groundbreaking integration of deep learning techniques in reinforcement learning algorithms.

The course is organized into 12 lectures with the following topics:

Lecture 01: Introduction to Reinforcement Learning and Decision-Making;
Lecture 02: Markov Decision Processes (MDPs);
Lecture 03: Dynamic Programming;
Lecture 04: Multi-Armed Bandits and Exploration vs Exploitation;
Lecture 05: Model-Free Evaluation;
Lecture 06: Model-Free Control;
Lecture 07: Reinforcement Learning with value function approximation;
Lecture 08: Introduction to Deep Reinforcement Learning;
Lecture 09: Policy Gradient;
Lecture 10: Introduction to Actor-Critic Approaches;
Lecture 11: Deep Actor-Critic Algorithms;
Lecture 12: Model-Based RL and Monte-Carlo Tree Search.

At the end of the course, the students will have developed the required theoretical and empirical knowledge about classic and modern reinforcement learning (RL) approaches. They will have a good knowledge of the core problems and goals of RL, its current state-of-the-art, and its future challenges. From a practical point of view, they will be able to design, implement, and carry out RL experiments using common tools, e.g., python libraries for RL and optimization. They will be able to conduct their M.Sc. theses on topics related to RL, and they will have good starting knowledge of RL for a future career in research or industry.

The evaluation consists of a written exam and 2 coding assignments.
The exam is in English and lasts 2 hours. There will be an exam at the end of the Summer Semester and one at the beginning of the Winter Semester. It will consist of open questions and/or quick application of algorithms by hand;
The coding assignments will be based on the technical skills developed in the exercise session, and will consist of autonomous implementation of algorithms, execution of experiments, and presentation of obtained results in a report of 2 pages. The two coding assignments will give, respectively, an additional bonus of 0.3 and 0.4 to the final grade of the written exam.

The course is organized into 12 lectures and 13 exercise sessions. The lectures will provide both theoretical and practical perspectives on the respective topics. The exercise session will consist of a quick refresh of the topic of the lecture and a guided coding session using the Python programming language to implement and run reinforcement learning experiments.

The main reference book for the course is:
- Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. MIT press, 2018. (http://incompleteideas.net/book/the-book.html).
Additional references are:
- Szepesvári, Csaba. "Algorithms for reinforcement learning." Synthesis lectures on artificial intelligence and machine learning 4.1 (2010): 1-103. (https://sites.ualberta.ca/~szepesva/papers/RLAlgsInMDPs.pdf);
- Deisenroth, Marc Peter, A. Aldo Faisal, and Cheng Soon Ong. Mathematics for machine learning. Cambridge University Press, 2020.