Lane Detection, Traffic Sign Recognition, and Scene Understanding

Module: Computer Vision | Difficulty: Premium

Bird's Eye View (BEV) Transform

where is predicted depth distribution.

Lane Detection Loss

BEVFormer Attention

where queries are BEV queries and keys/values are multi-view image features.

| Model | mAP | NDS | BEV | Latency | |-------|-----|-----|-----|---------| | PointPillars | 40.1 | 54.1 | No | 15ms | | BEVDet | 42.4 | 55.6 | Yes | 60ms | | BEVFormer | 51.7 | 63.0 | Yes | 100ms | | UniAD | 54.0 | 65.2 | Yes | 130ms | | SparseDrive | 55.8 | 66.5 | Yes | 80ms |

import torch
import torch.nn as nn
import torch.nn.functional as F

class LiftSplatShoot(nn.Module):
    def __init__(self, grid_config, bev_channels=64):
        super().__init__()
        self.grid_config = grid_config
        self.depth_net = nn.Sequential(
            nn.Conv2d(512, 256, 3, padding=1, bias=False),
            nn.BatchNorm2d(256), nn.ReLU(inplace=True),
            nn.Conv2d(256, grid_config['ddepth'], 1),
        )
        self.bev_proj = nn.Conv2d(
            grid_config['ddepth'], bev_channels, 1)

    def forward(self, features, camera_matrices):
        B, N, C, H, W = features.shape
        depth_logits = self.depth_net(
            features.view(B * N, C, H, W))
        depth = F.softmax(depth_logits, dim=1)
        depth = depth.view(B, N, -1, H, W)
        bev = self.bev_proj(depth.sum(dim=1))
        return bev

class LaneDetectionHead(nn.Module):
    def __init__(self, in_channels=64, num_lanes=4):
        super().__init__()
        self.seg_head = nn.Sequential(
            nn.Conv2d(in_channels, 32, 3, padding=1, bias=False),
            nn.BatchNorm2d(32), nn.ReLU(inplace=True),
            nn.Conv2d(32, num_lanes + 1, 1),
        )
        self.regression_head = nn.Sequential(
            nn.Conv2d(in_channels, 32, 3, padding=1, bias=False),
            nn.BatchNorm2d(32), nn.ReLU(inplace=True),
            nn.Conv2d(32, num_lanes * 2, 1),
        )

    def forward(self, bev_features):
        seg = self.seg_head(bev_features)
        reg = self.regression_head(bev_features)
        return seg, reg

Research Insight: Autonomous driving perception is shifting from single-task detectors to unified multi-task systems (UniAD) that jointly predict 3D boxes, lanes, maps, and motion forecasts. BEVFormer demonstrated that cross-attention between BEV queries and multi-view image features achieves strong 3D detection without LiDAR. The key challenge is real-time performance: safety-critical applications require < 100ms latency while processing 6+ camera views. Sparse attention and efficient BEV representations are active research areas for meeting these constraints.