Stanford AI Course Map 中文 GitHub ↗
← Back to the course map
Stanford · 2026—27 MS&E 235A Statistics & Optimization —

Related course · Statistics & Optimization

Markov Decision Processes (EE 283)

中文:马尔可夫决策过程

MDPs, value iteration, policy iteration, and bandit basics.

Draft translation: official facts are checked; wording may change after native review.

3 units Autumn 2026 Letter or Credit/No Credit —
Markov Decision Processes

Course positioning

MDPs, value iteration, policy iteration, and bandit basics. This page keeps the official record and entry points; check the current syllabus for details.

What it covers

This is a related-course page: the official description below is the main content.

Prerequisites

See the official description (related courses do not have a separate prerequisite write-up).

Assessment & workload

Check the current syllabus.

Who it's for

Anyone interested in Markov Decision Processes.

Study advice

Read the official description and course site before deciding.

Related courses

Official course description

Formulation and solution of sequential decision problems under uncertainty as a foundation for artificial intelligence, operations research, and economics. Finite-horizon, discounted, and average reward objectives. Optimization via value iteration, policy iteration, linear programming, and reinforcement learning algorithms. Semi-Markov decision processes. Multi-armed bandits and the Gittin's index theorem. Use of partial state information. Homework assignments involve a combination of analytic and computational exercises.