Content-Based Image Retrieval and Visual Search

Module: Computer Vision | Difficulty: Premium

Similarity Search

Average Pooling (AP) and GeM Pooling

where is a learnable parameter.

Recall@K

import torch
import torch.nn as nn
import torch.nn.functional as F

class GeMPooling(nn.Module):
    def __init__(self, p=3.0, eps=1e-6):
        super().__init__()
        self.p = nn.Parameter(torch.ones(1) * p)
        self.eps = eps

    def forward(self, x):
        x = x.clamp(min=self.eps).pow(self.p)
        return x.mean(dim=[2, 3]).pow(1.0 / self.p)

class ImageRetrievalModel(nn.Module):
    def __init__(self, backbone, embed_dim=512):
        super().__init__()
        self.backbone = backbone
        self.pool = GeMPooling()
        self.projection = nn.Linear(backbone.out_channels, embed_dim)

    def forward(self, x):
        features = self.backbone(x)
        pooled = self.pool(features)
        return F.normalize(self.projection(pooled), dim=1)

def compute_recall_at_k(dist_matrix, labels_q, labels_db, k=1):
    queries = dist_matrix.shape[0]
    correct = 0
    for i in range(queries):
        sorted_idx = dist_matrix[i].argsort()
        top_k = sorted_idx[:k]
        if labels_db[top_k] == labels_q[i]:
            correct += 1
    return correct / queries

Research Insight: Image retrieval has evolved from hand-crafted features (SIFT, SURF) to deep learning approaches that learn discriminative global descriptors. NetVLAD introduced differentiable VLAD aggregation, enabling end-to-end training for retrieval. Recent methods (CosPlace, Eagle) achieve state-of-the-art results by training with large-scale geolocated image collections, leveraging GPS tags as weak supervision. The challenge of long-tail distribution (rare landmarks have few images) remains, with self-supervised pretraining showing promise for improving recall on under-represented classes.