Instance Segmentation

Module: Computer Vision | Difficulty: Advanced

Mask R-CNN

Extends Faster R-CNN by adding a parallel mask prediction branch:

ROI Align

Solves the misalignment issue of ROI pooling via bilinear interpolation:

where is the bilinear interpolation kernel.

Panoptic Quality

Panoptic FPN

Combines semantic and instance segmentation with a unified feature pyramid:

import torch
import torch.nn as nn

class MaskHead(nn.Module):
    def __init__(self, in_ch=256, num_classes=80):
        super().__init__()
        self.conv = nn.Sequential(
            nn.Conv2d(in_ch, 256, 3, padding=1),
            nn.BatchNorm2d(256),
            nn.ReLU(inplace=True),
            nn.ConvTranspose2d(256, 256, 2, stride=2),
            nn.ReLU(inplace=True),
            nn.ConvTranspose2d(256, 256, 2, stride=2),
            nn.ReLU(inplace=True),
        )
        self.mask = nn.Conv2d(256, num_classes, 1)
    
    def forward(self, x):
        return self.mask(self.conv(x))

Key Takeaways

• Mask R-CNN predicts masks in parallel with classification
• ROI Align preserves spatial alignment for accurate masks
• Panoptic segmentation unifies things and stuff classes