Mask R-CNN and Modern Instance Segmentation

Module: Computer Vision | Difficulty: Premium

Mask R-CNN

where .

Panoptic Quality

Mask Classification (MaskFormer/Mask2Former)

where are predicted masks, are class predictions.

| Model | PQ | AP | Year | Paradigm | |-------|----|----|------|----------| | Mask R-CNN | 39.8 | 37.1 | 2017 | Instance | | Panoptic FPN | 40.9 | - | 2019 | Panoptic | | Mask2Former | 47.2 | 43.7 | 2021 | Mask classification | | OneFormer | 48.0 | 44.1 | 2022 | Unified | | SAM 2 | 50.1 | 46.2 | 2024 | Promptable |

import torch
import torch.nn as nn
import torch.nn.functional as F

class MaskHead(nn.Module):
    def __init__(self, in_channels=256, num_classes=80, num_masks=100):
        super().__init__()
        self.projection = nn.Linear(in_channels, in_channels)
        self.class_head = nn.Linear(in_channels, num_classes + 1)
        self.mask_head = nn.Sequential(
            nn.Linear(in_channels, in_channels),
            nn.ReLU(inplace=True),
            nn.Linear(in_channels, in_channels),
            nn.ReLU(inplace=True),
            nn.Linear(in_channels, num_masks),
        )

    def forward(self, roi_features):
        pooled = roi_features.mean(dim=[2, 3])
        proj = self.projection(pooled)
        classes = self.class_head(proj)
        masks = self.mask_head(proj)
        return classes, masks

class Mask2FormerDecoder(nn.Module):
    def __init__(self, in_dim=256, num_queries=100, num_heads=8):
        super().__init__()
        self.query_embed = nn.Embedding(num_queries, in_dim)
        self.cross_attn = nn.MultiheadAttention(
            in_dim, num_heads, batch_first=True)
        self.ffn = nn.Sequential(
            nn.Linear(in_dim, in_dim * 4),
            nn.GELU(), nn.Linear(in_dim * 4, in_dim))

    def forward(self, pixel_features):
        queries = self.query_embed.weight.unsqueeze(0).repeat(
            pixel_features.shape[0], 1, 1)
        attn_out, _ = self.cross_attn(
            queries, pixel_features, pixel_features)
        queries = queries + attn_out
        queries = queries + self.ffn(queries)
        masks = torch.einsum('bqd,bdhw->bqhw',
                             queries, pixel_features)
        return masks, queries

Research Insight: The shift from instance-specific prediction (Mask R-CNN) to mask classification (Mask2Former) was a paradigm change: instead of predicting a mask per class per instance, the model predicts a set of binary masks and assigns classes to them. This formulation unifies semantic, instance, and panoptic segmentation into a single framework. SAM 2 extended this to video with temporal tracking, enabling consistent mask propagation across frames. The mask classification paradigm also enables open-vocabulary segmentation when combined with CLIP features.