Deep Learning Transformers and Attention Syllabus

Contrast BERT and GPT in terms of directionality and typical use.

BERT is an encoder-only, bidirectional model best for understanding tasks (classification, NER, QA) via fine-tuning. GPT is a decoder-only, unidirectional (left-to-right) model best for generation and few-shot in-context learning.

What does 'autoregressive' mean for a language model?

The model factorizes the joint probability of a sequence as a product of conditional next-token probabilities $P(x) = \prod_t P(x_t \mid x_{<t})$, generating one token at a time conditioned on previously generated tokens.

What is T5 and its unifying framing of NLP tasks?

T5 (Text-to-Text Transfer Transformer) is an encoder-decoder model that casts every NLP task as text-to-text: input text in, target text out (e.g., translation, classification, summarization all become string-to-string), using a task prefix.

What pretraining objective does T5 use?

Span corruption (a denoising objective): contiguous spans of tokens are masked and replaced by unique sentinel tokens, and the model learns to generate the dropped spans as the target sequence.

Compare encoder-only, decoder-only, and encoder-decoder Transformer architectures.

Encoder-only (BERT): bidirectional, for understanding. Decoder-only (GPT): causal, for generation. Encoder-decoder (T5, original Transformer): encoder reads input bidirectionally, decoder generates output with cross-attention, ideal for seq2seq like translation.

What problem do subword tokenizers solve compared to word-level tokenization?

They handle out-of-vocabulary and rare words by breaking them into known subword units, keeping vocabulary size manageable while representing any string, including morphological variants and misspellings.

How does Byte-Pair Encoding (BPE) build its vocabulary?

Starting from characters/bytes, BPE iteratively counts adjacent symbol pairs and merges the most frequent pair into a new token, repeating until a target vocabulary size is reached. Merges are then applied greedily at inference.

How does WordPiece differ from BPE in choosing merges?

Instead of picking the most frequent pair, WordPiece (used by BERT) merges the pair that maximizes the likelihood of the training data, i.e., it selects the pair maximizing $\frac{\text{count}(xy)}{\text{count}(x)\,\text{count}(y)}$-style score.

What makes SentencePiece distinctive as a tokenizer?

SentencePiece treats the input as a raw stream (including spaces as a special symbol, e.g., '▁'), so it is language-agnostic and reversible, requiring no pre-tokenization. It can implement BPE or the unigram language model algorithm.

What are neural scaling laws for language models?

Empirical power-law relationships showing test loss decreases predictably as a power of model parameters $N$, dataset size $D$, and compute $C$: $$L(N) \approx \left(\frac{N_c}{N}\right)^{\alpha_N}$$ Loss falls smoothly and predictably as these scale up.

What is the key finding of the Chinchilla scaling law?

For a fixed compute budget, model size and training tokens should scale roughly equally; many earlier models were undertrained. Chinchilla recommends about 20 training tokens per parameter, and a smaller, longer-trained model outperformed larger undertrained ones.

What are 'emergent abilities' of large language models?

Capabilities (e.g., multi-step arithmetic, in-context reasoning) that are absent or near-random in smaller models but appear relatively abruptly once model scale crosses a threshold, and are not predictable by simple extrapolation of the loss curve alone.

What is in-context learning (ICL)?

The ability of a large model to perform a new task purely from examples and instructions given in the prompt at inference time, without any gradient updates or weight changes.

Distinguish zero-shot, one-shot, and few-shot prompting.

Zero-shot: only a task description, no examples. One-shot: exactly one demonstration example in the prompt. Few-shot: several demonstration examples in the prompt. More examples generally improve performance up to the context limit.

What is chain-of-thought (CoT) prompting?

Prompting the model to produce intermediate reasoning steps before the final answer (e.g., 'Let's think step by step'), which substantially improves performance on arithmetic, commonsense, and symbolic reasoning tasks, especially at large scale.

What is the difference between fine-tuning and pretraining?

Pretraining trains a model from scratch on massive unlabeled data with a self-supervised objective. Fine-tuning takes the pretrained model and further trains it (updating weights) on a smaller labeled or task-specific dataset to specialize it.

What is instruction tuning?

Fine-tuning a pretrained LLM on a diverse collection of tasks phrased as natural-language instructions with responses, so the model learns to follow instructions and generalizes to unseen instruction-formatted tasks in zero-shot settings.

Describe the three stages of RLHF (Reinforcement Learning from Human Feedback).

(1) Supervised fine-tuning on demonstration data; (2) train a reward model from human preference rankings of outputs; (3) optimize the policy against the reward model using RL, typically PPO, often with a KL penalty to stay close to the SFT model.

What is DPO (Direct Preference Optimization) and how does it differ from RLHF?

DPO directly optimizes the policy on preference pairs using a classification-style loss derived from the reward-model objective, eliminating the need to train a separate reward model or run RL/PPO. It is simpler and more stable while targeting the same preference-alignment goal.

What is Parameter-Efficient Fine-Tuning (PEFT) and why is it used?

PEFT adapts a large pretrained model by training only a small number of new or selected parameters while freezing most of the backbone. It drastically reduces memory, storage, and compute cost, and mitigates catastrophic forgetting, enabling many task-specific adapters per base model.

How does LoRA (Low-Rank Adaptation) work?

LoRA freezes pretrained weights $W$ and learns a low-rank update $\Delta W = BA$ where $B \in \mathbb{R}^{d \times r}$, $A \in \mathbb{R}^{r \times k}$, and $r \ll d,k$. The forward pass becomes $h = Wx + BAx$, so only $A$ and $B$ are trained, greatly reducing trainable parameters.

What is QLoRA?

QLoRA fine-tunes with LoRA adapters on top of a base model quantized to 4-bit (NF4), using techniques like double quantization and paged optimizers. It enables fine-tuning very large models on a single GPU with minimal quality loss.

Name three families of PEFT methods besides LoRA.

Adapter modules (small bottleneck layers inserted between Transformer sublayers), prompt/prefix tuning (learning soft continuous prompt vectors), and (IA)^3 / BitFit (scaling activations or training only bias terms).

What is Retrieval-Augmented Generation (RAG)?

RAG augments an LLM by retrieving relevant documents from an external knowledge source (often via vector similarity search) and conditioning generation on them, improving factual accuracy, enabling up-to-date knowledge, and reducing hallucination without retraining.

Outline the main steps of a RAG pipeline.

(1) Chunk and embed a document corpus into a vector store; (2) embed the user query; (3) retrieve top-$k$ similar chunks (e.g., by cosine similarity); (4) insert retrieved context into the prompt; (5) the LLM generates an answer grounded in that context.

How does a Vision Transformer (ViT) adapt Transformers to images?

ViT splits an image into fixed-size patches (e.g., $16\times16$), linearly embeds each patch into a token, adds positional embeddings and a learnable [class] token, then feeds the sequence into a standard Transformer encoder for classification.

What is a key data requirement of ViTs compared to CNNs?

ViTs lack the built-in inductive biases (locality, translation equivariance) of CNNs, so they need large-scale pretraining data (e.g., JFT-300M) to match or exceed CNNs; with limited data CNNs often perform better.

What is CLIP and how is it trained?

CLIP (Contrastive Language-Image Pretraining) jointly trains an image encoder and text encoder on image-caption pairs with a contrastive objective, maximizing cosine similarity of matched image-text embeddings and minimizing it for mismatched pairs within a batch (symmetric InfoNCE loss).

How does CLIP enable zero-shot image classification?

Class names are turned into text prompts (e.g., 'a photo of a {class}') and encoded; the image embedding is compared by cosine similarity to each class-text embedding, and the highest-similarity class is chosen, with no task-specific training.

What is the contrastive InfoNCE-style loss objective used in contrastive pretraining?

For a matched pair among $N$ candidates, minimize $$\mathcal{L} = -\log \frac{\exp(\text{sim}(z_i, z_i^{+})/\tau)}{\sum_{j=1}^{N} \exp(\text{sim}(z_i, z_j)/\tau)}$$ where $\tau$ is a temperature and $\text{sim}$ is cosine similarity.

What are common strategies for fusing modalities in multimodal Transformer architectures?

Early fusion (concatenate modality tokens into one Transformer with cross-modal self-attention), late fusion (separate encoders combined at the end), and cross-attention fusion (one modality attends to another, e.g., Flamingo's gated cross-attention injecting vision into a language model).

How do multimodal LLMs typically connect a vision encoder to a language model?

A frozen or trained vision encoder (often ViT/CLIP) produces image features that a projection/adapter module (e.g., a linear layer or Q-Former as in BLIP-2, or an MLP as in LLaVA) maps into the LLM's token embedding space so images become 'tokens' the LLM reads.

What problem does FlashAttention solve and how?

It removes the $O(n^{2})$ memory bottleneck of materializing the full attention matrix. FlashAttention is an IO-aware, exact attention algorithm that tiles $Q$, $K$, $V$ into SRAM blocks and uses online softmax to compute attention without ever writing the $n\times n$ matrix to HBM, giving large speedups and linear memory.

Is FlashAttention an approximation of attention?

No. FlashAttention computes exact attention (identical outputs to standard attention). Its gains come from reducing memory reads/writes to slow HBM via kernel fusion, tiling, and recomputation in the backward pass, not from approximating the softmax.

What is sparse attention and how does it reduce complexity?

Sparse attention restricts each token to attend to a subset of positions (e.g., local windows, strided, or global tokens as in Longformer/BigBird) instead of all $n$, reducing complexity from $O(n^{2})$ toward $O(n)$ or $O(n\log n)$ while retaining most modeling power.

How does linear attention achieve $O(n)$ complexity?

Linear attention replaces the softmax with a kernel feature map $\phi(\cdot)$ so attention becomes $\phi(Q)\big(\phi(K)^{T}V\big)$. By computing $\phi(K)^{T}V$ first (associativity), it avoids the $n\times n$ matrix, giving $O(n\,d^{2})$ time and memory linear in sequence length.

What is Multi-Query Attention (MQA) / Grouped-Query Attention (GQA)?

MQA shares a single Key/Value head across all Query heads, and GQA shares K/V among groups of query heads. Both shrink the KV cache and speed up autoregressive inference with little quality loss, interpolating between full multi-head (per-head KV) and single-head KV.

Why is the KV cache used during autoregressive generation, and what is its memory cost?

During generation, previously computed Key and Value vectors are cached so each new token only computes attention against stored K/V instead of recomputing them. Its memory grows linearly with sequence length and number of layers/heads, and often becomes the dominant inference memory cost, motivating MQA/GQA.