CNN Architectures and Convolution Operations

Module: Computer Vision | Difficulty: Premium

Discrete Convolution in 2D

For an input feature map and kernel of size :

Output Size Formula

where is input size, is kernel size, is padding, is stride.

Receptive Field Growth

For a stack of layers with kernel size and stride :

Depthwise Separable Convolution

Splits standard convolution into depthwise and pointwise:

import torch
import torch.nn as nn

class ConvBlock(nn.Module):
    def __init__(self, in_c, out_c, kernel=3, stride=1, padding=1):
        super().__init__()
        self.conv = nn.Conv2d(in_c, out_c, kernel, stride, padding, bias=False)
        self.bn = nn.BatchNorm2d(out_c)
        self.relu = nn.ReLU(inplace=True)
    def forward(self, x):
        return self.relu(self.bn(self.conv(x)))

class DepthwiseSeparable(nn.Module):
    def __init__(self, in_c, out_c, kernel=3, stride=1, padding=1):
        super().__init__()
        self.depthwise = nn.Conv2d(in_c, in_c, kernel, stride, padding, groups=in_c, bias=False)
        self.pointwise = nn.Conv2d(in_c, out_c, 1, bias=False)
        self.bn = nn.BatchNorm2d(out_c)
        self.relu = nn.ReLU(inplace=True)
    def forward(self, x):
        return self.relu(self.bn(self.pointwise(self.depthwise(x))))

def count_params(model):
    return sum(p.numel() for p in model.parameters() if p.requires_grad)

Research Insight: The shift from hand-designed architectures to Neural Architecture Search (NAS) revealed that human intuitions about efficient designs are often suboptimal. EfficientNet's compound scaling demonstrated that depth, width, and resolution must be co-optimized. Recent work on ConvNeXt shows that with modern training recipes (larger kernels, fewer activation functions, layer normalization), pure convolutional networks can match or exceed transformer-based vision models.