Stanford AI 课程地图 EN GitHub ↗
← 返回课程地图
Stanford · 2026—27 MS&E 319 统计与优化 ★★★★☆

核心课程 · 统计与优化

高效生成模型

Efficient Generative Models

高效生成模型:在推理深度、算力、延迟、内存、数据之间做权衡,重点是 LLM。

3 学分 2026 秋 Letter or Credit/No Credit ★★★★☆
生成模型效率LLM系统

课程定位

MS&E 319 研究生成模型在计算约束下的算法问题。课程讨论目标、架构、数据 pipeline、推理过程和服务系统如何被算力/延迟/内存/数据塑造,并以 LLM 为主要案例;学生做一次 presentation,再完成 survey 或研究项目。

主要内容

  1. 计算约束:推理深度、算力、延迟、内存、数据
  2. 目标与架构:scaling、稀疏、混合专家
  3. 数据 pipeline:质量、配比、合成数据
  4. 推理:采样、speculative decoding、test-time compute
  5. Serving:批处理、缓存、调度
  6. 算法问题:约束下的新问题
  7. presentation + survey/project

课程大纲 / 课件

#主题 / 讲者
01Overview of the Course and LLMs
02A Review of LLM Architecture and Training
03Making Compute Go Further
04Mixtures of Experts
05Efficient Attention: KV Caching, Sparse Attention, Compression
06Efficient Attention: Compact KV-Cache Representations
07Positional Embeddings and Recurrent Memory
08Language Model Quantization I: Quantizing Vectors
09Language Model Quantization II: Quantizing Matrices
10Speculative Decoding and Efficient Inference
11Post-training I: SFT, LoRA, RLHF
12Post-training II: RLHF and DPO
13Distillation and Recursive Self-Improvement
14Risks and Vulnerabilities of Generative AI (I)
15Risks and Vulnerabilities of Generative AI (II)
16Where Do We Go from Here?

Autumn 2026 lecture outline;slides 在课程页逐讲链接。

先修

ML/LLM 基础;算法与系统意识。

考核与工作量

一次 presentation + survey 或 research project。具体以当季官网为准。

适合谁

想做推理效率、生成模型算法、ML 系统的人;研究生。

学习建议

配合 CS 349E 学工程实现,配合 CS 336 学模型训练。

相关课程

官方课程描述(英文原文)

As generative models become central to computing, their progress increasingly depends on algorithms that trade reasoning depth, computational cost, latency, memory, and data. This course studies those tradeoffs with a primary focus on modern language models. We examine how objectives, architectures, data pipelines, inference procedures, and serving systems are shaped by computational constraints, and how these constraints open new algorithmic questions. Students give one presentation and complete either a survey or a research project. May be repeated for credit.