Course positioning
Taught by Thomas Icard, CS 221M is one of the few courses that systematically teaches mechanistic interpretability. It emphasizes causal methods rather than 'looking at pictures', covering probing, steering, causal abstraction, SAEs, and circuits, with guest lectures from academic and industry interpretability labs.
What it covers
- Probing: linear probes, representation analysis, limitations
- Steering: activation interventions, direction vectors, causal tracing
- Causal abstraction: causal graphs, interventions, abstraction mappings
- Sparse autoencoders: dictionary learning, feature decomposition, monosemanticity
- Circuits: attention heads, MLP neurons, composition
- LLM internals: fact recall, reasoning, deception
- Interpretability and safety: using it for auditing and alignment
Prerequisites
Deep learning/LLM basics; probability and linear algebra; having read about Transformers helps.
Assessment & workload
Assignments plus a project/paper reproduction; research- and discussion-oriented. Check the current syllabus for details.
Who it's for
People who want to do interpretability, AI safety, or model auditing; researchers who want to understand LLM internals.
Study advice
Read Anthropic's 'Towards Monosemanticity' and 'Scaling Monosemanticity' first, plus the causal-abstraction papers.
Related courses
Official course description
What is the internal structure of modern neural networks and how can we study it? This course provides a broad and deep introduction to interpretability, the subfield of machine learning concerned with understanding precisely how models process information and why they produce the outputs they do. We will cover topics such as probing, steering, causal abstraction, and sparse autoencoders, with a particular emphasis on causal methods and large language models. The course will include guest lectures from leading interpretability labs across academia and industry.