Deep Semantic Segmentation

Module: Computer Vision | Difficulty: Advanced

Fully Convolutional Networks (FCN)

Replace fully connected layers with 1×1 convolutions to produce spatial output:

Atrous (Dilated) Convolution

Expands the receptive field without increasing parameters:

where the dilation rate expands the sampling grid.

ASPP (Atrous Spatial Pyramid Pooling)

Parallel dilated convolutions at multiple rates capture multi-scale context:

DeepLab v3+ Architecture

Encoder: ResNet + ASPP. Decoder: Upsample + low-level features.

Mean IoU Metric

import torch
import torch.nn as nn

class ASPP(nn.Module):
    def __init__(self, in_ch, out_ch, rates=[6, 12, 18]):
        super().__init__()
        modules = [nn.Sequential(
            nn.Conv2d(in_ch, out_ch, 1, padding=0, dilation=1),
            nn.BatchNorm2d(out_ch),
            nn.ReLU(inplace=True)
        )]
        for r in rates:
            modules.append(nn.Sequential(
                nn.Conv2d(in_ch, out_ch, 3, padding=r, dilation=r),
                nn.BatchNorm2d(out_ch),
                nn.ReLU(inplace=True)
            ))
        self.convs = nn.ModuleList(modules)
        self.project = nn.Sequential(
            nn.Conv2d(out_ch * (len(rates) + 1), out_ch, 1),
            nn.BatchNorm2d(out_ch),
            nn.ReLU(inplace=True),
            nn.Dropout(0.5)
        )
    
    def forward(self, x):
        feats = [conv(x) for conv in self.convs]
        return self.project(torch.cat(feats, dim=1))

Key Takeaways

• Dilated convolution increases receptive field without losing resolution
• ASPP captures multi-scale context efficiently
• DeepLab v3+ achieves state-of-the-art on PASCAL VOC and Cityscapes