NeRF Extensions

Module: Computer Vision | Difficulty: Advanced

Instant-NGP

Multiresolution hash encoding:

3D Gaussian Splatting

Represent scene as set of 3D Gaussians:

Differentiable Rasterization

where

Training Speed Comparison

| Method | Time to Quality | |--------|----------------| | NeRF | Hours | | Instant-NGP | Minutes | | Gaussian Splatting | Minutes |

import torch
import torch.nn as nn

class GaussianSplatting(nn.Module):
    def __init__(self, num_gaussians=100000):
        super().__init__()
        self.means = nn.Parameter(torch.randn(num_gaussians, 3))
        self.scales = nn.Parameter(torch.ones(num_gaussians, 3) * 0.01)
        self.rotations = nn.Parameter(torch.randn(num_gaussians, 4))
        self.features = nn.Parameter(torch.randn(num_gaussians, 48))
        self.opacity = nn.Parameter(torch.zeros(num_gaussians, 1))
    
    def forward(self, rays):
        # Simplified: compute color from Gaussians along rays
        pass

Key Takeaways

• Instant-NGP uses hash encoding for 100x faster training
• 3D Gaussian Splatting enables real-time rendering
• These methods achieve quality comparable to original NeRF