Human Pose Estimation and Keypoint Detection

Module: Computer Vision | Difficulty: Premium

Heatmap Regression

For each keypoint , predict a Gaussian heatmap:

OKS (Object Keypoint Similarity)

where is Euclidean distance, is object scale, is per-keypoint normalization.

PCK (Percentage of Correct Keypoints)

| Approach | Method | Speed | Accuracy | Multi-person | |----------|--------|-------|----------|--------------| | Top-Down | HRNet | 5 fps | 75.1 AP | Separate detector | | Top-Down | ViTPose | 12 fps | 77.6 AP | Transformer | | Bottom-Up | OpenPose | 25 fps | 65.7 AP | Grouping required | | Bottom-Up | HigherHRNet | 10 fps | 70.2 AP | Heatmap grouping |

import torch
import torch.nn as nn

class SimplePoseNet(nn.Module):
    def __init__(self, num_keypoints=17):
        super().__init__()
        self.backbone = nn.Sequential(
            nn.Conv2d(3, 64, 7, stride=2, padding=3, bias=False),
            nn.BatchNorm2d(64),
            nn.ReLU(inplace=True),
            self._make_layer(64, 256, 3),
            self._make_layer(256, 256, 4),
            self._make_layer(256, 256, 6),
            self._make_layer(256, 256, 3),
        )
        self.head = nn.Sequential(
            nn.Conv2d(256, 256, 3, padding=1, bias=False),
            nn.BatchNorm2d(256),
            nn.ReLU(inplace=True),
            nn.Conv2d(256, num_keypoints, 1),
        )

    def _make_layer(self, in_c, out_c, blocks):
        layers = [nn.Conv2d(in_c, out_c, 1, bias=False),
                  nn.BatchNorm2d(out_c), nn.ReLU(inplace=True)]
        for _ in range(blocks):
            layers.extend([
                nn.Conv2d(out_c, out_c, 3, padding=1, bias=False),
                nn.BatchNorm2d(out_c), nn.ReLU(inplace=True),
            ])
        return nn.Sequential(*layers)

    def forward(self, x):
        features = self.backbone(x)
        heatmaps = self.head(features)
        return heatmaps

def decode_heatmaps(heatmaps, threshold=0.5):
    N, K, H, W = heatmaps.shape
    coords = []
    for k in range(K):
        hm = heatmaps[0, k]
        if hm.max() < threshold:
            coords.append([0, 0])
            continue
        y, x = torch.unravel_index(hm.argmax(), hm.shape)
        coords.append([x.item(), y.item()])
    return torch.tensor(coords)

Research Insight: HRNet maintains high-resolution representations throughout the network, avoiding the information loss from encoder-decoder architectures. ViTPose demonstrated that Vision Transformers with simple decoder heads achieve state-of-the-art pose estimation, benefiting from global self-attention that captures long-range spatial dependencies. The key remaining challenge is 3D pose estimation from monocular images, where depth ambiguity and self-occlusion make the problem ill-posed without temporal or multi-view cues.