Course positioning
CS 349E (EE 339) is taught by Mark Horowitz, Fredrik Kjolstad, and others — a hard-core mix of systems, algorithms, and hardware. It discusses how to serve large models on heterogeneous machines with low latency, high throughput, and low cost, covering quantization, distributed serving, performance modeling, kernel programming, and AI compilers.
What it covers
- Model optimization: quantization, pruning, distillation, speculative decoding
- Distributed serving: tensor parallelism, pipelines, prefill/decode disaggregation
- AI hardware: GPUs/TPUs/accelerators, the memory wall
- Performance modeling: roofline, bandwidth/compute/latency
- Kernel programming: CUDA/Triton, fusion, tiling
- AI compilers: graph optimization, operator selection, code generation
- Project: end-to-end inference optimization
Prerequisites
Computer architecture, operating systems, ML basics; strong programming.
Assessment & workload
Assignments plus a project, focused on implementation and measurement. Check the current syllabus for details.
Who it's for
Graduate students/engineers working on inference performance, AI chips, or compilers; systems people going deeper into ML.
Study advice
Brush up on CS 217/CS 149 first. Be ready to run benchmarks, not just read paper numbers.
Related courses
Official course description
A graduate-level course focused on principles and techniques for designing and building large-scale disaggregated AI inference systems. The course explores architectural, algorithmic, system-level, and kernel implementation techniques for serving models with low latency, high throughput, and low cost on heterogeneous machines. Topics include model optimization techniques, distributed serving, AI hardware and performance modelling, kernel programming, and AI compilers. Class website: cs349e.github.io