Stanford AI Course Map 中文 GitHub ↗
← Back to the course map
Stanford · 2026—27 CS 349E ML Systems & Hardware ★★★★★

Core course · ML Systems & Hardware

Efficient ML Inference at Scale (EE 339)

中文:大规模高效 ML 推理

Efficient inference at scale: model optimization, distributed serving, AI hardware, kernels, and compilers.

Draft translation: official facts are checked; wording may change after native review.

3 units Autumn 2026 Letter or Credit/No Credit ★★★★★
InferencePerformanceHardwareCompilers

Course positioning

CS 349E (EE 339) is taught by Mark Horowitz, Fredrik Kjolstad, and others — a hard-core mix of systems, algorithms, and hardware. It discusses how to serve large models on heterogeneous machines with low latency, high throughput, and low cost, covering quantization, distributed serving, performance modeling, kernel programming, and AI compilers.

What it covers

  1. Model optimization: quantization, pruning, distillation, speculative decoding
  2. Distributed serving: tensor parallelism, pipelines, prefill/decode disaggregation
  3. AI hardware: GPUs/TPUs/accelerators, the memory wall
  4. Performance modeling: roofline, bandwidth/compute/latency
  5. Kernel programming: CUDA/Triton, fusion, tiling
  6. AI compilers: graph optimization, operator selection, code generation
  7. Project: end-to-end inference optimization

Prerequisites

Computer architecture, operating systems, ML basics; strong programming.

Assessment & workload

Assignments plus a project, focused on implementation and measurement. Check the current syllabus for details.

Who it's for

Graduate students/engineers working on inference performance, AI chips, or compilers; systems people going deeper into ML.

Study advice

Brush up on CS 217/CS 149 first. Be ready to run benchmarks, not just read paper numbers.

Related courses

Official course description

A graduate-level course focused on principles and techniques for designing and building large-scale disaggregated AI inference systems. The course explores architectural, algorithmic, system-level, and kernel implementation techniques for serving models with low latency, high throughput, and low cost on heterogeneous machines. Topics include model optimization techniques, distributed serving, AI hardware and performance modelling, kernel programming, and AI compilers. Class website: cs349e.github.io