Stanford AI 课程地图 EN GitHub ↗
← 返回课程地图
Stanford · 2026—27 CS 349D ML 系统与硬件 ★★★★☆

核心课程 · ML 系统与硬件

AI 推理基础设施

AI Inference Infrastructure

AI 推理基础设施:compound AI systems 在云上如何部署、调度、扩展。

3 学分 2027 春 Letter or Credit/No Credit ★★★★☆
推理基础设施云系统

课程定位

CS 349D 由 Christos Kozyrakis 讲授,是一门研究型 seminar,聚焦“推理”而不是训练。课程认为未来的 AI 系统是 compound systems(多模型 + 检索 + 工具 + 传统软件),并讨论支撑它们的云基础设施、调度和架构模式。

主要内容

  1. 推理 serving:批处理、连续批处理、KV cache
  2. Compound AI systems:多组件编排、数据流
  3. 云基础设施:弹性、多租户、成本
  4. 调度与路由:模型路由、优先级、SLA
  5. 延迟/吞吐/成本:权衡与优化
  6. 工业实践:Databricks、OpenAI、NVIDIA 等 guest lecture
  7. quarter-long project

课程大纲 / 课件

#主题 / 讲者
01Introduction
02Parallelism
03Serving Engines
04Programming Assignment Meeting 1
05Batching
06Disaggregation
07Speculative Decoding
08SGLang (Ying Sheng, RadixArk)
09Context Management
10Efficient Attention
11Hybrid LLMs
12Mixture of Experts
13Programming Assignment Meeting 2
14Load Balancing & Routing
15Agentic Workflows
16Memory
17Reinforcement Learning
18Multi-tenancy
19Project Presentations

Spring 2026 研究型 seminar;每讲以论文讨论为主,reading 在 schedule 页。

先修

系统/体系结构/ML 基础;有分布式系统经验更好。

考核与工作量

论文讨论 + guest lectures + 2–3 人小组项目。具体以当季官网为准。

适合谁

想做推理平台、AI 基础设施、云系统的人;系统背景的研究生。

学习建议

配合 CS 349E 学单机/集群性能,配合 CS 349F 学网络。项目尽量基于真实 serving 栈。

相关课程

官方课程描述(英文原文)

The emergence of foundation models has revolutionized AI applications, but the most powerful AI systems today are increasingly compound systems - compositions of multiple models, retrievers, tools, and traditional software components. This research seminar explores the intersection of cloud computing infrastructure and compound AI systems, covering both the infrastructure required to build and deploy these systems at scale and the architectural patterns that make them effective. Students will participate in guest lectures from leading experts across academia and industry, including leaders of companies like Databricks, OpenAI, Foundry, NVIDIA, Microsoft, and more, read and discuss papers, and complete a quarter-long project in groups of 2-3.