Course positioning
CS 224N is Stanford's flagship NLP course and the first stop into LLMs for many people. It starts with word representations and language models, then moves to Transformers, pre-training, fine-tuning, and RLHF, combining mathematical derivations with substantial implementation.
What it covers
- Word representations: word2vec, GloVe, subwords/tokenization
- Language models: n-grams, RNNs/LSTMs, attention
- Transformers: self-attention, positional encoding, architecture details
- Pre-trained models: BERT, GPT, T5, scaling laws
- Adaptation and alignment: fine-tuning, prompting, RLHF, DPO
- NLP tasks: syntax, semantics, QA, machine translation, information extraction
- Final project: usually an LLM/agent research or application project
Prerequisites
Calculus and linear algebra; one of CS 124, CS 221, or CS 229.
Assessment & workload
Typically 4–5 assignments plus a midterm and a final project; the assignments involve substantial PyTorch implementation. Check the current syllabus for details.
Who it's for
Required for NLP/LLM; people working on agents, RAG, alignment, or multimodal language models; engineers moving from classical NLP to LLMs.
Study advice
Take CS 229 first, or at least understand softmax, cross-entropy, and backpropagation. If you want to train LLMs from scratch, follow this with CS 336.
Related courses
Official course description
Methods for processing human language information and the underlying computational properties of natural languages. Focus on deep learning approaches: understanding, implementing, training, debugging, visualizing, and extending neural network models for a variety of language understanding tasks. Exploration of natural language tasks ranging from simple word level and syntactic processing to coreference, question answering, and machine translation. Examination of representative papers and systems and completion of a final project applying a complex neural network model to a large-scale NLP problem. Prerequisites: calculus and linear algebra; CS124, CS221, or CS229.