Monocular Depth Estimation and Depth Prediction

Module: Computer Vision | Difficulty: Premium

Scale Ambiguity

The absolute scale is ambiguous without known camera motion.

Photometric Loss (Self-Supervised)

where are sampled source pixels projected via depth and pose.

Depth Evaluation Metrics

import torch
import torch.nn as nn

class DepthDecoder(nn.Module):
    def __init__(self, encoder_channels=[256, 512, 1024, 2048]):
        super().__init__()
        self.reduce = nn.ModuleList([
            nn.Conv2d(c, 256, 1) for c in encoder_channels])
        self.decoder = nn.ModuleList([
            nn.Sequential(
                nn.Conv2d(256, 256, 3, padding=1, bias=False),
                nn.BatchNorm2d(256), nn.ReLU(inplace=True),
            ) for _ in range(4)])
        self.head = nn.Conv2d(256, 1, 3, padding=1)

    def forward(self, features):
        x = self.reduce[-1](features[-1])
        for i in range(len(features) - 2, -1, -1):
            x = nn.functional.interpolate(
                x, scale_factor=2, mode='bilinear',
                align_corners=True)
            x = x + self.reduce[i](features[i])
            x = self.decoder[i](x)
        return self.head(x)

class MonoDepthNet(nn.Module):
    def __init__(self):
        super().__init__()
        import torchvision.models as models
        encoder = models.resnet50(pretrained=True)
        self.encoder_layers = nn.ModuleList([
            nn.Sequential(encoder.conv1, encoder.bn1, encoder.relu),
            nn.Sequential(encoder.maxpool, encoder.layer1),
            encoder.layer2, encoder.layer3, encoder.layer4,
        ])
        self.decoder = DepthDecoder([64, 256, 512, 1024, 2048])

    def forward(self, x):
        features = []
        for layer in self.encoder_layers:
            x = layer(x)
            features.append(x)
        return self.decoder(features)

def compute_depth_metrics(pred, gt, mask=None):
    if mask is None:
        mask = gt > 0
    pred = pred[mask]
    gt = gt[mask]
    abs_rel = (torch.abs(pred - gt) / gt).mean()
    sq_rel = ((pred - gt) ** 2 / gt).mean()
    rmse = torch.sqrt(((pred - gt) ** 2).mean())
    return {'abs_rel': abs_rel.item(),
            'sq_rel': sq_rel.item(), 'rmse': rmse.item()}

Research Insight: Monocular depth estimation has made remarkable progress through self-supervised learning, where depth is learned jointly with camera pose from unpaired video sequences. The key insight of MonoDepth2 was to model the output distribution as a mixture, handling static scenes where the photometric assumption fails. Recent methods (Depth Anything V2, UniDepth) leverage large-scale synthetic data and foundation model features to predict metric depth without scene-specific calibration, approaching the quality of LiDAR sensors.