Stanford AI Course Map 中文 GitHub ↗
← Back to the course map
Stanford · 2026—27 CS 234 Foundations ★★★★☆

Core course · Foundations

Reinforcement Learning

中文:强化学习

The core reinforcement learning course: MDPs, value iteration, policy gradients, exploration, with deep RL in the assignments.

Draft translation: official facts are checked; wording may change after native review.

Not offered in the 2026–27 ExploreCourses listing; this page is based on the most recent offering.

3 units Not scheduled in 2026–27 Letter or Credit/No Credit ★★★★☆
RLDecision-makingRobotics

Course positioning

CS 234 covers the main line of RL from tabular methods to deep RL, with an emphasis on how autonomous systems make decisions through trial and error. It fits sequential decision-making in robotics, games, recommendation, and healthcare, and is a prerequisite for CS 224R/CS 332.

What it covers

  1. MDP basics: returns, Bellman equations, policy evaluation
  2. Dynamic programming: value iteration, policy iteration
  3. Model-free methods: Monte Carlo, TD, Q-learning, SARSA
  4. Function approximation: linear approximation, deep Q-networks
  5. Policy gradients: REINFORCE, actor-critic, PPO
  6. Exploration: ε-greedy, UCB, Thompson sampling
  7. Deep RL and offline RL: stability issues in practice
  8. Project: usually train an agent on a control or game task

Prerequisites

Fluent Python; CS 229 or equivalent ML background; linear algebra and probability.

Assessment & workload

Typically 3–4 assignments plus a midterm and a project; at least one deep RL assignment. Check the current syllabus for details.

Who it's for

Robotics, RL research, game AI, and recommender systems; anyone working on embodied AI.

Study advice

Master MDPs and dynamic programming before deep RL. For the exploration part, read Chapters 2–3 of Sutton & Barto.

Related courses

Official course description

To realize the dreams and impact of AI requires autonomous systems that learn to make good decisions. Reinforcement learning is one powerful paradigm for doing so, and it is relevant to an enormous range of tasks, including robotics, game playing, consumer modeling and healthcare. This class will briefly cover background on Markov decision processes and reinforcement learning, before focusing on some of the central problems, including scaling up to large domains and the exploration challenge. One key tool for tackling complex RL domains is deep learning and this class will include at least one homework on deep reinforcement learning. Prerequisites: proficiency in python, CS 229 or equivalents or permission of the instructor; linear algebra, basic probability.