课程定位
CS 329A 由 Anmol Chowdhery 和 Azalia Mirhoseini 讲授,聚焦“agent 如何通过与环境/自身交互持续变强”。课程以论文讨论和原创研究项目为主,还会邀请学术和工业界讲者,适合已经决定做 agent 研究的人。
主要内容
- LLM 自我改进:constitutional AI、self-critique、self-play
- Verifier 与搜索:learned verifier、best-of-n、树搜索
- Test-time compute:推理时扩展、思维链、并行采样
- 工具与检索:tool use、retrieval、多模态 web 交互
- 多步推理与规划:长程任务分解、编排框架
- 评估与 orchestration:鲁棒 eval、失败模式
- 应用:coding agent、STEM 研究助手、机器人
课程大纲 / 课件
Autumn 2025 schedule;seminar 以论文阅读和 guest lecture 为主,slides 未公开。
先修
CS 224N 或 CS 229S;熟练 Python 和 LLM API。
考核与工作量
论文阅读/讨论 + 原创 research project + 讲者讲座。具体以当季官网为准。
适合谁
想做 agent 研究、alignment、推理/规划方向的研究生;已经有一定 LLM 经验的人。
学习建议
先读 ReAct、Reflexion、Tree of Thoughts、Self-Rewarding LM 等论文。项目要有清晰的 baseline 和 eval。
相关课程
官方课程描述(英文原文)
This graduate seminar course covers the latest techniques and applications of AI agents that can continuously improve themselves through interaction with themselves and the environment. The course will start with self-improvement techniques for LLMs, such as constitutional AI, using learned/domain-specific verifiers, scaling test-time compute, and combining search with LLMs. We will then discuss the latest research in augmenting LLMs with tool use and retrieval techniques, and orchestrating AI capabilities with multimodal web interaction. We will next discuss multi-step reasoning and planning problems for agentic workflows, and the challenges in building robust evaluation and orchestration frameworks. Industry applications that will be discussed include coding agents, research assistants in STEM, robotics and more. Students will work on an original research project in this area, discuss the suggested readings in each class, and learn from invited academic and industry speakers. Prerequisites: CS224N or CS229S; Fluency in Python programming and using large language model APIs.