Vision Benchmarks

Module: Computer Vision | Difficulty: Beginner

Standard Benchmarks

ImageNet

• 1.28M training images, 1000 classes
• Top-1 and Top-5 accuracy

COCO

• 330K images, 80 object categories
• Detection: mAP@0.5:0.95
• Segmentation: mAP, mask AP

Cityscapes

• 5000 fine-annotated images
• 19 semantic classes
• Metrics: mIoU, pixel accuracy

Evaluation Metrics

import torch

def accuracy(output, target, topk=(1, 5)):
    with torch.no_grad():
        maxk = max(topk)
        batch_size = target.size(0)
        _, pred = output.topk(maxk, 1, True, True)
        pred = pred.t()
        correct = pred.eq(target.view(1, -1).expand_as(pred))
        res = []
        for k in topk:
            correct_k = correct[:k].reshape(-1).float().sum(0)
            res.append(correct_k / batch_size)
        return res

def mAP(predictions, ground_truths, iou_threshold=0.5):
    aps = []
    for cls in range(num_classes):
        # Compute precision-recall curve
        # Calculate AP using VOC or COCO method
        ap = compute_ap(cls_preds, cls_gts, iou_threshold)
        aps.append(ap)
    return sum(aps) / len(aps)

Key Takeaways

• ImageNet is the standard for image classification
• COCO defines evaluation for detection and segmentation
• Understanding metrics is essential for fair comparison