CSCI 5932: Deep Reinforcement Learning

Instructor: Alessandro Roncone

Email:alessandro.roncone@colorado.edu

Explore the principles of deep reinforcement learning (DRL) and its applications to robotics. Learn to design, train, and deploy DRL models for robotic tasks, emphasizing control, perception, and decision-making in complex environments. Through hands-on projects and simulations, develop the skills to tackle real-world challenges in autonomous robotics using state-of-the-art algorithms and tools.Same as CSCI 4932 and ROBO 5329.

Prerequisites

Intro to Robotics (CSCI 5202 or ROBO 5000), Advanced Robotics (CSCI 5302 or ROBO 5302), or Machine Learning (CSCI 5622).

Recommended restrictions:Ìý

  • Proficiency in Python and ideally LaTeX
  • Linear Algebra and Calculus
  • Probability and Statistics
  • Foundations of Machine Learning
  • Numerical Optimization

Grading

Graduate students’ grading will have more weight on the final project.

Topics Covered

  • Markov Decision Processes
  • Partially Observable Markov Decision Processes
  • Q-Learning
  • Policy Gradients
  • Actor-Critic
  • Deep Q-Networks
  • Model-based planning and policy learning
  • Exploration-Exploitation
  • Inverse Reinforcement Learning
  • Distributed Reinforcement Learning

Course Outcome

By the end of the class students should be able to:

  • Articulate the key features and components of reinforcement learning and its distinction from other machine learning tools.
  • Given a specific application, formulate it formally from an RL perspective (in terms of the state space, action space, dynamics, and reward model), state what algorithm is best suited for addressing it and justify the answer.
  • Implement in code common RL algorithms (as assessed by the homework assignments).
  • Develop sufficient proficiency to perform research-level work on the topic.

We will have 4 homework assignments and a course project. There will also be a short quiz every lecture. The assignments are intended primarily to get students started with resources for studying RL questions and algorithms, and hence are expected to be fairly light. The course project is the main outcome of this course.