Prerequisites

Before diving into CNNs, you should be comfortable with:

• Neural Network Fundamentals: Forward/backward pass, activation functions, gradient descent (see Tutorial 21)
• Linear Algebra: 2D/3D matrix operations, convolution as matrix multiplication
• Image Representation: RGB channels, pixel values (0-255), spatial dimensions (H x W x C)
• PyTorch Basics: Tensors, nn.Module, DataLoader, GPU acceleration

Learning Objectives

By the end of this tutorial, you will be able to:

1. Compute convolution operations and understand how kernels extract visual features
2. Calculate output dimensions given input size, kernel size, stride, and padding
3. Compare max pooling, average pooling, and global average pooling
4. Explain how ResNet skip connections solve the vanishing gradient problem
5. Perform transfer learning with pre-trained CNN models
6. Implement a complete CNN in PyTorch for image classification
7. Understand the feature hierarchy from edges to objects

Convolutional Neural Networks — Complete Guide

CNNs exploit the spatial structure of images through two key principles: local connectivity (each neuron connects to a small region) and weight sharing (same filter applied everywhere). This yields parameters instead of for fully connected layers.

Convolution Operation

The discrete 2D convolution (cross-correlation in practice) slides a learnable kernel over the input:

Pooling

Pooling reduces spatial dimensions, providing translation invariance and reducing computation.

ResNet and Skip Connections

The residual connection addresses the degradation problem. Instead of learning directly, learn the residual :

PyTorch Implementation

CNN Architecture Comparison

Real-World Applications

Common Mistakes & How to Avoid Them

Key Formulas Reference

Interview Questions

Practice Exercise

import torch
import torch.nn as nn
import torchvision
import torchvision.transforms as transforms

# Task 1: Implement a CNN with these specifications:
# - Conv layer 1: 3 -> 32 channels, 3x3 kernel, same padding
# - Conv layer 2: 32 -> 64 channels, 3x3 kernel, same padding
# - Conv layer 3: 64 -> 128 channels, 3x3 kernel, same padding
# - Each conv block: Conv -> BatchNorm -> ReLU -> MaxPool(2)
# - Classifier: GAP -> Linear(128, 64) -> ReLU -> Dropout(0.3) -> Linear(64, 10)

class YourCNN(nn.Module):
    def __init__(self):
        super().__init__()
        # Your implementation here
        pass

    def forward(self, x):
        # Your implementation here
        pass

# Task 2: Train for 50 epochs with:
# - Adam optimizer, lr=1e-3
# - CosineAnnealingLR scheduler
# - Data augmentation: RandomCrop(32, padding=4), RandomHorizontalFlip()

# Task 3: Achieve >85% test accuracy on CIFAR-10
# Bonus: Add a residual connection between conv blocks 2 and 3

Key Takeaways

What to Learn Next

Advanced Topics

Depthwise Separable Convolutions

Standard convolutions compute across all spatial and channel dimensions simultaneously. Depthwise separable convolutions factorize this into two steps:

1. Depthwise convolution: One filter per input channel (spatial filtering only)
2. Pointwise convolution: 1x1 conv to combine channels

Parameter reduction: From to . For : standard = 1,179,648 params vs separable = 144,128 params (~8x reduction).

Receptive Field Analysis

The receptive field is the region of the input that influences a particular output neuron. Understanding receptive fields is crucial for designing architectures:

• Single 3x3 conv: receptive field = 3x3
• Two stacked 3x3 convs: receptive field = 5x5
• Three stacked 3x3 convs: receptive field = 7x7 (same as one 7x7 with fewer params)

Formula:

where is kernel size and is stride at layer .

Feature Visualization

Understanding what CNNs learn is crucial for debugging and trust:

• Layer 1: Gabor-like edge detectors, color blobs
• Layer 2: Corners, textures, simple patterns
• Layer 3: Object parts (eyes, wheels, leaves)
• Layer 4: Whole objects (faces, cars, animals)
• Layer 5: Scene-level concepts (faces, specific objects)

Techniques: Deconvolution (Zeiler & FERGUS, 2014), Grad-CAM (visualizing which regions drive predictions), feature inversion (reconstructing inputs from activations).

Comparison Table

Further Reading

• LeCun, Y. et al. (1998). "Gradient-based learning applied to document recognition." — The original LeNet paper that started CNNs.
• He, K. et al. (2016). "Deep Residual Learning for Image Recognition." — The ResNet paper that enabled training 152+ layer networks.
• Simonyan, K. & Zisserman, A. (2014). "Very Deep Convolutional Networks for Large-Scale Image Recognition." — VGGNet, establishing 3x3 as the standard kernel size.
• Howard, A. et al. (2017). "MobileNets: Efficient CNNs for Mobile Vision Applications." — Depthwise separable convolutions for edge deployment.
• Tan, M. & Le, Q. (2019). "EfficientNet: Rethinking Model Scaling for CNNs." — Compound scaling strategy.
• CS231n: Convolutional Networks for Visual Recognition (Stanford) — The definitive free course on CNNs and computer vision.

Quick Reference Cheat Sheet