Deep Learning Convolutional Neural Networks Syllabus

What is a bottleneck residual block in ResNet-50/101/152?

A bottleneck stacks $1\times1$ (reduce channels), $3\times3$, then $1\times1$ (restore channels) convolutions with a skip connection. The $1\times1$ layers cut computation so deeper networks stay efficient compared with two $3\times3$ layers.

What problem motivated ResNet, and what did it observe about plain deep networks?

The degradation problem: adding more layers to a plain network increased training error (not just from overfitting). ResNet showed that residual connections let deeper networks match or beat shallow ones, winning ImageNet 2015.

What is compound scaling in EfficientNet?

EfficientNet scales network depth $d$, width $w$, and input resolution $r$ together with a single coefficient $\phi$: $d=\alpha^{\phi}$, $w=\beta^{\phi}$, $r=\gamma^{\phi}$, subject to $\alpha\cdot\beta^{2}\cdot\gamma^{2}\approx 2$, so FLOPs scale by about $2^{\phi}$. This balanced scaling beats scaling any single dimension.

What baseline building block does EfficientNet use, and what does its constraint $\alpha\cdot\beta^{2}\cdot\gamma^{2}\approx2$ mean?

EfficientNet-B0 is built from MBConv (mobile inverted bottleneck) blocks with squeeze-and-excitation. The constraint ensures that for each unit increase in $\phi$ the total FLOPs roughly double, since width and resolution contribute quadratically to compute.

Contrast two-stage (region-based) detectors with one-stage detectors.

Two-stage detectors (R-CNN family) first propose candidate regions, then classify/refine them—more accurate but slower. One-stage detectors (YOLO, SSD) predict boxes and classes directly over a dense grid in a single pass—faster, historically slightly less accurate on small objects.

Describe the R-CNN pipeline and its main drawback.

R-CNN uses selective search to generate ~2000 region proposals, warps each, runs a CNN per region for features, then classifies with SVMs and regresses boxes. Its drawback is extreme slowness: the CNN runs separately on every proposal, with no shared computation.

How does Fast R-CNN improve on R-CNN?

Fast R-CNN runs the CNN once over the whole image, then uses RoI pooling to extract fixed-size features per proposal from the shared feature map, and trains classification and bounding-box regression jointly with a multi-task loss—much faster and more accurate.

What is the key innovation of Faster R-CNN?

Faster R-CNN replaces selective search with a learnable Region Proposal Network (RPN) that shares convolutional features with the detector and predicts objectness and box offsets over anchors, giving a near real-time end-to-end trainable two-stage detector.

What is RoI Pooling, and how does RoI Align differ?

RoI Pooling divides each region proposal into a fixed grid and max-pools features, but quantizes coordinates, causing misalignment. RoI Align (from Mask R-CNN) avoids quantization by using bilinear interpolation at sampled points, giving precise, sub-pixel-accurate features—critical for segmentation.

How does YOLO frame object detection, and what does it predict per grid cell?

YOLO treats detection as a single regression: it divides the image into an $S\times S$ grid and each cell predicts $B$ bounding boxes (x, y, w, h, confidence) plus $C$ class probabilities. Confidence is $P(\text{object})\times \text{IoU}$. It is very fast (single forward pass).

How does SSD (Single Shot Detector) detect objects at multiple scales?

SSD attaches detection heads to several feature maps of decreasing resolution, using default (anchor) boxes of multiple aspect ratios at each location. Shallow high-resolution maps detect small objects; deep low-resolution maps detect large objects—all in one pass.

What are anchor boxes and why are they used?

Anchor boxes are predefined reference boxes of various scales and aspect ratios placed densely across the feature map. The network predicts class scores and offsets relative to each anchor rather than absolute coordinates, making detection of multiple objects and shapes per location tractable.

Define Intersection over Union (IoU) and its formula.

IoU measures overlap between predicted box $A$ and ground-truth box $B$: $$\text{IoU}=\frac{|A\cap B|}{|A\cup B|}.$$ It ranges from 0 (no overlap) to 1 (perfect). A threshold (commonly 0.5) decides positive matches.

Describe the Non-Maximum Suppression (NMS) algorithm.

NMS removes duplicate detections: sort boxes by confidence, select the highest, remove all remaining boxes with IoU above a threshold (e.g., 0.5) relative to it, and repeat on the rest. It keeps one box per object. Soft-NMS decays scores instead of hard removal.

How is mean Average Precision (mAP) computed for object detection?

For each class, compute Average Precision as the area under the precision–recall curve (matches decided by an IoU threshold), then average AP over all classes. COCO reports mAP averaged over IoU thresholds from 0.5 to 0.95 in steps of 0.05.

What is DETR and how does it eliminate hand-designed components?

DETR (DEtection TRansformer) treats detection as direct set prediction: a CNN backbone feeds a transformer encoder–decoder that outputs a fixed set of predictions from learned object queries. It removes anchors and NMS, replacing them with bipartite matching and self-attention.

What loss and matching does DETR use to train?

DETR uses the Hungarian algorithm to find a bipartite matching between predicted and ground-truth objects (a one-to-one assignment), then applies a set loss combining classification cross-entropy and box loss ($\ell_1$ + generalized IoU). Unmatched predictions are trained toward a 'no object' class.

What are object queries in DETR?

Object queries are a fixed set of $N$ learned embeddings fed to the transformer decoder; each attends to image features and produces one detection (class + box). $N$ is set larger than the typical number of objects, with extras predicting 'no object'.

Distinguish semantic segmentation, instance segmentation, and panoptic segmentation.

Semantic segmentation labels every pixel with a class but does not separate object instances. Instance segmentation detects and masks each object separately (only 'things'). Panoptic segmentation unifies both: per-pixel class labels plus instance IDs for countable objects, with 'stuff' regions labeled semantically.

What defines a Fully Convolutional Network (FCN) for segmentation?

An FCN replaces the fully connected layers of a classifier with convolutions, so it outputs a spatial class map instead of a single vector, accepting arbitrary input sizes. Upsampling (transposed convolution) restores resolution to produce dense per-pixel predictions.

What is transposed convolution (deconvolution) and its role in segmentation?

Transposed convolution is a learnable upsampling operation that increases spatial resolution by spreading each input value over a kernel-sized output region (the gradient operation of convolution). FCNs and U-Net use it to map coarse feature maps back to full-resolution masks.

What are skip connections in FCN (e.g., FCN-8s) used for?

They fuse coarse, deep semantic features with finer, shallow high-resolution features (via elementwise addition after upsampling), recovering spatial detail lost to downsampling. FCN-8s combines predictions from pool3, pool4, and the final layer for sharper boundaries.

Describe the U-Net architecture.

U-Net has a symmetric encoder–decoder ('U' shape): a contracting path of conv+pooling captures context, an expanding path of transposed convs restores resolution, and skip connections concatenate encoder feature maps to the matching decoder level, preserving spatial detail. Designed for biomedical image segmentation.

Why are skip connections in U-Net concatenations rather than additions, and what do they provide?

U-Net concatenates encoder features onto decoder features at each resolution so the decoder can combine high-resolution localization information with upsampled semantic context, producing precise boundaries. Concatenation preserves both signals rather than merging them additively.

What loss functions are common for semantic segmentation, and why is Dice loss useful?

Pixel-wise cross-entropy is standard. Dice loss, based on the Dice coefficient $$\text{Dice}=\frac{2|X\cap Y|}{|X|+|Y|},$$ directly optimizes overlap and handles class imbalance well (e.g., small foreground regions), so it is popular in medical imaging.

How does Mask R-CNN extend Faster R-CNN?

Mask R-CNN adds a third branch that outputs a binary segmentation mask for each RoI in parallel with the classification and box-regression branches, and replaces RoI Pooling with RoI Align for pixel-accurate features—enabling instance segmentation.

Why does Mask R-CNN predict masks per class independently rather than with a softmax across classes?

It predicts a separate binary mask for each class and relies on the classification branch to select which mask to use. This decouples mask prediction from class competition, avoiding inter-class competition and improving mask quality.

What is a pretrained backbone and why is transfer learning with one effective?

A backbone is a feature-extraction network (e.g., ResNet, VGG, EfficientNet) pretrained on a large dataset like ImageNet. Its early layers learn general features (edges, textures) transferable across tasks, so fine-tuning on a smaller dataset converges faster and generalizes better than training from scratch.

Contrast feature extraction versus fine-tuning when using a pretrained backbone.

Feature extraction freezes the backbone weights and trains only a new head—fast and good for small/similar datasets. Fine-tuning unfreezes some or all backbone layers and continues training at a low learning rate—better when the target dataset is larger or differs from the pretraining domain.

What is a Feature Pyramid Network (FPN) and why is it used with backbones in detection?

FPN builds a multi-scale feature pyramid by combining a bottom-up backbone pathway with a top-down pathway and lateral skip connections, producing semantically strong features at every resolution. It improves detection of objects across scales (used in Faster R-CNN, Mask R-CNN, RetinaNet).