Saliency Detection

Module: Computer Vision | Difficulty: Intermediate

Saliency Map

KL Divergence Loss

NSS (Normalized Saliency)

AUC (Area Under ROC)

import torch
import torch.nn as nn

class SaliencyNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.backbone = nn.Sequential(
            nn.Conv2d(3, 64, 3, padding=1), nn.ReLU(True),
            nn.Conv2d(64, 128, 3, 2, 1), nn.ReLU(True),
            nn.Conv2d(128, 256, 3, 2, 1), nn.ReLU(True),
        )
        self.head = nn.Sequential(
            nn.Conv2d(256, 128, 3, padding=1), nn.ReLU(True),
            nn.ConvTranspose2d(128, 64, 4, 2, 1), nn.ReLU(True),
            nn.ConvTranspose2d(64, 1, 4, 2, 1), nn.Sigmoid(),
        )
    
    def forward(self, x):
        feat = self.backbone(x)
        return self.head(feat)

Key Takeaways

• Saliency detection predicts where humans look in images
• KL divergence and NSS are standard evaluation metrics
• Saliency models improve image compression and advertisement placement