课程定位
CS 324 是 2023 年推出的 foundation model 课程,覆盖 GPT-3、CLIP、Stable Diffusion 等模型的原理、系统和社会影响。课程有工业界讲者,重点是项目而不是推导,适合想快速建立 FM 全局认知的人。
主要内容
- FM 基础:定义、历史、缩放规律
- 模型与算法:Transformer、多模态、扩散
- 适配:提示、微调、LoRA、RLHF
- 系统:分布式训练、推理、成本
- 伦理与社会影响:偏见、版权、滥用、监管
- 应用:文本、图像、代码、科学
- quarter-long project:研究 FM 能力或构建应用
先修
深度学习基础(CS 224N/CS 230/CS 231N 级)。
考核与工作量
以 quarter-long final project 为主,可能配合作业/阅读。具体以当季官网为准。
适合谁
想了解 foundation model 生态的产品/研究/战略人员;想找 FM 应用方向的学生。
学习建议
课程内容偏 2023,注意结合最新论文。项目建议选一个窄而深的问题。
相关课程
官方课程描述(英文原文)
Foundation models (FMs) are transforming the landscape of AI in research and industry. Such models (e.g., GPT-3, CLIP, Stable Diffusion) are trained on large amounts of broad data and are adaptable to a wide range of downstream tasks. In this course, students will learn fundamentals behind the models and algorithms, systems and infrastructure, and ethics and societal impacts of foundation models, with an emphasis on gaining hands-on experience and identifying real-world use-cases for FMs. Students will hear from speakers in industry working on foundation models in the wild. The main class assignment will be a quarter-long final project, involving either researching the capabilities of FMs or building an FM-powered application.