This course is entirely based on the instrumental course of Sergey Levine taught at Berkeley CS189. Hence full credit should be given to Sergey Levine and his team for creating such great materials. This course follows its structure and homeworks except for the final project. It is one of the optional courses of the IASD M2 program

This course will assume some familiarity with reinforcement learning, numerical optimization, and machine learning. For introductory material on RL and MDPs, see the Berkeley CS188 EdX course, starting with Markov Decision Processes I, as well as Chapters 3 and 4 of Sutton & Barto. It is worth also checking the introductory course on machine learning Berkeley CS189.

There will be five homeworks. For each homework, we will post a PDF on the front page and starter code that will be posted on Github.

We will post slides on the front page after each lecture.

All homeworks should be done individually.

For the final project, you may work in groups of up to three people. Each group will submit a report. The expectations for the project scope will increase depending on the number of students in each group, and for groups of two or three, we will also expect a short paragraph to explain the role of each group member along with the final report. From past experience, groups of two tend to be the most effective, though you may work in a group of three or alone. Groups larger than three are not permitted without special permission from the course staff.

All assignments must be turned in via Moodle on time. We will allow a total of five late days cumulatively. We will not make any additional allowances for late assignments: the late days are intended to provide for exceptional circumstances, and students should avoid using them unless absolutely necessary. Any assignments that are submitted late (with insufficient late days remaining) will not be graded.

Late days may not be used for quizzes, final project proposals, final project milestone reports, final project reports, or any of the project peer review reports, only for the five homeworks.

• Homework: 50% (10% per HW x 5 HWs)
• Final Project: 50%