Convolutional Neural Networks for Vision

Module: Computer Vision | Difficulty: Advanced

The Convolution Operation

CNNs exploit the spatial structure of images through weight sharing and local connectivity. The discrete convolution operation is the mathematical foundation:

CNN Feature Extraction Pipeline

Input32×32×3Conv13×3 kernel32×32×32ReLU activation921K paramsPool116×16×322×2 maxConv23×3 kernel16×16×6436.9K paramsPool28×8×64Flatten4096FC10 classesHierarchical Feature LearningLayer 1: Edges, ColorsLayer 2: Textures, CornersLayer 3: Parts, PatternsLayer 4-5: Objects, ScenesResidual Connection: y = F(x) + x → Enables training of 100+ layer networksSkip gradients directly through identity mapping → Solves vanishing gradient problem

Mathematical Foundations

Convolution as Matrix Operation

A convolution with kernel of size on input of size :

The number of parameters per filter: (bias).

Receptive Field Growth

The receptive field of layer :

where is the kernel size and is the stride at layer .

Pooling Theory

Max Pooling:

Global Average Pooling:

Code Example: Custom CNN with Residual Connections

import torch
import torch.nn as nn

class ResidualBlock(nn.Module):
    """Residual block with skip connection."""
    
    def __init__(self, channels, stride=1):
        super().__init__()
        self.conv1 = nn.Conv2d(channels, channels, 3, stride, 1, bias=False)
        self.bn1 = nn.BatchNorm2d(channels)
        self.conv2 = nn.Conv2d(channels, channels, 3, 1, 1, bias=False)
        self.bn2 = nn.BatchNorm2d(channels)
        self.relu = nn.ReLU(inplace=True)
        
        self.shortcut = nn.Sequential()
        if stride != 1 or channels != channels:
            self.shortcut = nn.Sequential(
                nn.Conv2d(channels, channels, 1, stride, bias=False),
                nn.BatchNorm2d(channels)
            )
    
    def forward(self, x):
        residual = self.shortcut(x)
        out = self.relu(self.bn1(self.conv1(x)))
        out = self.bn2(self.conv2(out))
        out += residual  # Skip connection
        return self.relu(out)

class VisionCNN(nn.Module):
    """CNN with residual connections for image classification."""
    
    def __init__(self, num_classes=10):
        super().__init__()
        self.prep = nn.Sequential(
            nn.Conv2d(3, 64, 3, 1, 1, bias=False),
            nn.BatchNorm2d(64),
            nn.ReLU(inplace=True)
        )
        self.layer1 = self._make_layer(64, 2, stride=1)
        self.layer2 = self._make_layer(128, 2, stride=2)
        self.layer3 = self._make_layer(256, 2, stride=2)
        self.gap = nn.AdaptiveAvgPool2d(1)
        self.fc = nn.Linear(256, num_classes)
    
    def _make_layer(self, channels, blocks, stride):
        layers = [ResidualBlock(channels, stride)]
        for _ in range(1, blocks):
            layers.append(ResidualBlock(channels))
        return nn.Sequential(*layers)
    
    def forward(self, x):
        x = self.prep(x)
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.gap(x)
        x = x.view(x.size(0), -1)
        return self.fc(x)

# Model summary
model = VisionCNN(num_classes=10)
params = sum(p.numel() for p in model.parameters())
print(f"Parameters: {params:,}")

Summary

CNNs learn hierarchical features — edges → textures → parts → objects
Weight sharing reduces parameters dramatically (vs. fully connected)
Residual connections enable training of 100+ layer networks
Receptive field grows with depth — deeper layers see larger regions
Transfer learning from ImageNet pretrained models is standard practice

Convolutional Neural Networks for Vision — From Theory to State-of-the-Art