SIFT, SURF, ORB, and Learned Feature Descriptors

Module: Computer Vision | Difficulty: Premium

Scale-Space Representation

where .

Difference of Gaussians (DoG)

SIFT Keypoint Localization

Keypoints are extrema in scale space. Refine position via Taylor expansion:

Hamming Distance for Binary Descriptors

| Descriptor | Type | Dim | Speed | Accuracy | |------------|------|-----|-------|----------| | SIFT | Float | 128 | Slow | High | | SURF | Float | 64 | Fast | High | | ORB | Binary | 256 | Very Fast | Medium | | HardNet | Float | 128 | Fast | Very High | | SuperPoint | Float | 256 | Fast | Very High |

import cv2
import numpy as np

def detect_and_compute(img, method='sift'):
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    if method == 'sift':
        detector = cv2.SIFT_create(nfeatures=1000)
    elif method == 'orb':
        detector = cv2.ORB_create(nfeatures=1000)
    elif method == 'akaze':
        detector = cv2.AKAZE_create()
    else:
        raise ValueError(f"Unknown method: {method}")
    keypoints, descriptors = detector.detectAndCompute(gray, None)
    return keypoints, descriptors

def match_features(desc1, desc2, method='bf', ratio=0.75):
    if method == 'bf':
        if desc1.dtype == np.uint8:
            matcher = cv2.BFMatcher(cv2.NORM_HAMMING)
        else:
            matcher = cv2.BFMatcher(cv2.NORM_L2)
        raw = matcher.knnMatch(desc1, desc2, k=2)
    else:
        flann = cv2.FlannBasedMatcher(
            dict(algorithm=1, trees=5),
            dict(checks=50))
        raw = flann.knnMatch(desc1, desc2, k=2)
    good = [m for m, n in raw if m.distance < ratio * n.distance]
    return good

Research Insight: Classical features like SIFT rely on hand-crafted scale-space theory, while learned descriptors (SuperPoint, R2D2) optimize end-to-end for matching performance. SuperPoint uses a self-supervised pretraining pipeline (Homographic Adaptation) to generate training data without manual annotation. The convergence of classical geometry (epipolar constraints, PnP) with learned features enables robust sparse 3D reconstruction that is both accurate and efficient at scale.