Deep Learning Convolutional Neural Networks Syllabus

Convolutional Neural Networks syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Convolutional Neural Networks in Deep Learning, not a summary of it.

1. 1

   Convolution Fundamentals

   4 topics

   • The Convolution Operation
   • Pooling Layers
   • Feature Maps and Channels
   • Parameter Sharing and Sparsity
2. 2

   Classic CNN Architectures

   5 topics

   • LeNet and AlexNet
   • VGGNet
   • GoogLeNet and Inception
   • ResNet and Residual Connections
   • EfficientNet and Scaling Laws
3. 3

   Object Detection

   4 topics

   • Region-Based Detectors
   • Single-Shot Detectors (YOLO, SSD)
   • Anchor Boxes and NMS
   • Detection Transformers (DETR)
4. 4

   Image Segmentation

   4 topics

   • Semantic vs Instance Segmentation
   • Fully Convolutional Networks
   • U-Net Architecture
   • Mask R-CNN
5. 5

   Transfer Learning and Fine-Tuning

   3 topics

   • Pretrained Backbones
   • Feature Extraction vs Fine-Tuning
   • Domain Adaptation

Convolutional Neural Networks flashcards for Deep Learning

22 of 50 cards from the Convolutional Neural Networks deck — real questions with worked answers.

1. What is the discrete 2D convolution (cross-correlation) operation used in CNNs, expressed as a formula?

   For input $I$ and kernel $K$, the output feature map is $$S(i,j)=(I*K)(i,j)=\sum_{m}\sum_{n} I(i+m,\,j+n)\,K(m,n).$$ Deep-learning libraries implement cross-correlation (no kernel flip), though it is called convolution.

2. Give the formula for the output spatial dimension of a convolution layer in terms of input size $W$, kernel size $F$, padding $P$, and stride $S$.

   $$O=\left\lfloor \frac{W-F+2P}{S}\right\rfloor + 1.$$ Same padding keeps $O=W$ (for stride 1) using $P=\frac{F-1}{2}$; valid padding uses $P=0$.

3. What is 'same' vs 'valid' padding?

   Valid padding applies no padding, so the output shrinks by $F-1$ each dimension. Same padding adds zeros so the output spatial size equals the input (at stride 1), requiring $P=\frac{F-1}{2}$ for odd $F$.

4. Define the receptive field of a neuron in a CNN and how it grows with depth.

   The receptive field is the region of the input that influences a given activation. It grows with each layer; stacking two $3\times3$ convs gives an effective $5\times5$ receptive field, and stride/pooling multiply the growth rate.

5. What is the purpose of a pooling layer, and what are the two most common types?

   Pooling downsamples feature maps to reduce spatial size and computation and to add small translation invariance. The two common types are max pooling (takes the maximum in each window) and average pooling (takes the mean).

6. How does max pooling behave during the backward pass?

   The gradient is routed only to the input element that was the maximum in each pooling window (argmax); all other positions in the window receive zero gradient. Average pooling instead distributes the gradient equally across the window.

7. What is global average pooling (GAP) and why is it used instead of fully connected layers?

   GAP averages each entire feature map to a single value, producing a vector of length equal to the number of channels. It has no learnable parameters, reduces overfitting, and enforces correspondence between channels and classes (used in NiN, GoogLeNet, ResNet).

8. Define a feature map and a channel in a convolutional layer.

   A feature map is the 2D activation produced by convolving one filter over the input, responding to a specific pattern. A channel is one such map; a layer's output is a stack of $C$ channels (a tensor of shape $H\times W\times C$).

9. How many parameters does a convolution layer have with $C_{in}$ input channels, $C_{out}$ filters, and kernel size $F\times F$ (with bias)?

   $$\#\text{params}=(F\times F\times C_{in}+1)\times C_{out}.$$ The $+1$ per filter is the bias term. Crucially this is independent of the input spatial size.

10. Explain parameter sharing in CNNs and the benefit it provides.

    The same filter weights are reused at every spatial location, so a feature detector applies everywhere. This drastically reduces parameters versus a fully connected layer and gives translation equivariance: shifting the input shifts the feature map correspondingly.

11. Explain sparse connectivity (local receptive fields) in convolution.

    Each output unit connects only to a small local patch of the input (the kernel size) rather than the whole input. This yields sparse interactions, fewer parameters, and lower computation than dense (fully connected) layers.

12. What is a $1\times1$ convolution and what is it used for?

    A $1\times1$ convolution mixes information across channels at each spatial location, acting as a per-pixel fully connected layer. It is used for dimensionality reduction/expansion of channels (bottlenecks in Inception, ResNet) and to add nonlinearity cheaply.

13. Describe the architecture of LeNet-5 and what it was designed for.

    LeNet-5 (LeCun, 1998) for handwritten digit recognition: two conv+subsampling (avg pool) stages followed by fully connected layers, using $5\times5$ convolutions and tanh/sigmoid activations, ending in a 10-way output. It is the classic template of alternating conv and pooling layers.

14. What key innovations did AlexNet (2012) introduce that revived deep learning?

    AlexNet used ReLU activations, GPU training, dropout, local response normalization, overlapping max pooling, and data augmentation. It won ImageNet 2012 with a top-5 error of about 15.3%, far ahead of the runner-up, launching the deep CNN era.

15. What is the design principle of VGGNet?

    VGG stacks many small $3\times3$ convolutions (stride 1, same padding) with $2\times2$ max pooling, doubling channels after each pool. Using stacked $3\times3$ filters instead of large ones gives the same receptive field with fewer parameters and more nonlinearity. VGG-16 and VGG-19 are common variants.

16. Why are two stacked $3\times3$ convolutions preferred over one $5\times5$ convolution?

    They share the same $5\times5$ effective receptive field but use fewer parameters ($2\cdot 9C^2=18C^2$ vs $25C^2$ per channel pair) and insert an extra nonlinearity, increasing representational power. Three $3\times3$ convs equal one $7\times7$.

17. What is the Inception module in GoogLeNet and its key idea?

    An Inception module runs parallel branches ($1\times1$, $3\times3$, $5\times5$ convolutions and $3\times3$ max pooling) and concatenates their outputs, letting the network capture multiple scales. $1\times1$ convolutions are placed before the larger convs as bottlenecks to cut computation.

18. What are auxiliary classifiers in GoogLeNet (Inception v1)?

    Auxiliary classifiers are small classification heads attached to intermediate layers during training. They inject additional gradient signal to combat vanishing gradients and act as regularizers; their losses are weighted (e.g., 0.3) and they are removed at inference.

19. State the residual learning formulation used in ResNet.

    Instead of learning a direct mapping $H(x)$, a residual block learns $F(x)=H(x)-x$ and outputs $$y=F(x)+x.$$ The identity skip connection makes it easy to learn near-identity mappings, enabling training of very deep networks (e.g., 152 layers).

20. How do residual (skip) connections mitigate the vanishing gradient problem?

    The identity shortcut provides an additive gradient path so that $\frac{\partial y}{\partial x}=1+\frac{\partial F}{\partial x}$, ensuring gradients flow directly to earlier layers even when $\frac{\partial F}{\partial x}$ is small, preventing degradation as depth increases.

21. What is a bottleneck residual block in ResNet-50/101/152?

    A bottleneck stacks $1\times1$ (reduce channels), $3\times3$, then $1\times1$ (restore channels) convolutions with a skip connection. The $1\times1$ layers cut computation so deeper networks stay efficient compared with two $3\times3$ layers.

22. What problem motivated ResNet, and what did it observe about plain deep networks?

    The degradation problem: adding more layers to a plain network increased training error (not just from overfitting). ResNet showed that residual connections let deeper networks match or beat shallow ones, winning ImageNet 2015.

See more Convolutional Neural Networks flashcards →

Planning Convolutional Neural Networks for Deep Learning

Convolutional Neural Networks (Deep Learning) FAQ