Robot Learning
Reinforcement learning, imitation learning and foundation models for robots.
ROB-410 Year 4 3 credits 12 lessons · 6 h 8 min
Offered by HSM University · Robotics
What you'll learn
- Explain when learning is preferable to hand-engineered robot behavior and estimate the data and compute a learning approach needs.
- Formulate robot tasks as Markov decision processes, including observations, actions, rewards, discounting and partial observability.
- Apply dynamic programming, value iteration and LQR, and explain how learned models enable model-based control.
- Implement and analyze value-based reinforcement learning, including Q-learning and the ideas behind deep Q-networks.
- Derive and apply policy-gradient and actor-critic methods, including baselines, advantages and clipped PPO objectives.
- Design rewards that are correct and learnable, recognize reward hacking and apply safety filters and constraints.
- Use parallel simulation, domain randomization, curricula and teacher-student training to transfer policies from simulation to real robots.
- Train behavior-cloning policies and explain distribution shift, compounding error and how DAgger and its variants correct them.
- Explain inverse reinforcement learning and select generative policies such as Diffusion Policy and ACT for multimodal demonstrations.
- Choose and evaluate learned visual representations, multi-sensor fusion and world models for robot control.
- Describe the architecture of vision-language-action foundation models and critically evaluate claims about them.
- Evaluate learned policies with sound statistics, design layered safety and staged deployment, and address Canadian and US data and AI governance.
Course content
12 lessons · 6 h 8 min total · final exam
- 1 Why robots learn 35 min · quiz
- 2 Markov decision processes 31 min · quiz
- 3 Dynamic programming and model-based control 30 min · quiz
- 4 Reinforcement learning I: value-based methods 30 min · quiz
- 5 Reinforcement learning II: policy gradients and actor-critic methods 31 min · quiz
- 6 Reward design and safety in reinforcement learning 31 min · quiz
- 7 Simulation for robot learning 29 min · quiz
- 8 Imitation learning I: behavior cloning and DAgger 29 min · quiz
- 9 Imitation learning II: inverse reinforcement learning and generative policies 30 min · quiz
- 10 Learning perception and representations 28 min · quiz
- 11 Foundation models and vision-language-action policies 30 min · quiz
- 12 Deployment, evaluation and capstone: a learning pipeline for pick-and-place 34 min · quiz
- E Final exam 20 questions
Requirements
Recommended background. You can start any course at any time.
Description
Reinforcement learning, imitation learning and foundation models for robots.
Part of the Science stream, typically taken in year 4.