3D Computer Vision

Module: Computer Vision | Difficulty: Advanced

Stereo Vision

Disparity to Depth

where is focal length, is baseline, and is disparity.

Triangulation

Given two camera poses, the 3D point is:

Point Cloud Processing

A point cloud contains 3D coordinates and features.

Iterative Closest Point (ICP)

where is the nearest neighbor of in the target cloud.

import numpy as np
from scipy.spatial import cKDTree

def icp(source, target, max_iter=20, tol=1e-6):
    R = np.eye(3)
    t = np.zeros(3)
    tree = cKDTree(target)
    
    for _ in range(max_iter):
        src_transformed = (R @ source.T).T + t
        dists, indices = tree.query(src_transformed)
        
        matched_target = target[indices]
        centroid_src = src_transformed.mean(axis=0)
        centroid_tgt = matched_target.mean(axis=0)
        
        H = (src_transformed - centroid_src).T @ (matched_target - centroid_tgt)
        U, S, Vt = np.linalg.svd(H)
        R_new = Vt.T @ U.T
        t_new = centroid_tgt - R_new @ centroid_src
        
        if np.linalg.norm(t_new - t) < tol:
            break
        R, t = R_new, t_new
    
    return R, t

Key Takeaways

• Stereo vision recovers depth from two viewpoint images
• ICP aligns point clouds iteratively
• Depth estimation can be monocular, stereo, or LiDAR-based