Perceptual Quality Metrics and Image Quality Enhancement

Module: Computer Vision | Difficulty: Premium

SSIM

LPIPS

Natural Image Statistics (NIQE)

| Model | LIVE | CSIQ | TID2013 | Type | |-------|------|------|---------|------| | PSNR | 0.87 | 0.82 | 0.68 | Full-ref | | SSIM | 0.95 | 0.91 | 0.78 | Full-ref | | LPIPS | 0.96 | 0.93 | 0.85 | Full-ref | | MUSIQ | 0.94 | 0.92 | 0.87 | No-ref | | CLIPIQA | 0.95 | 0.94 | 0.89 | No-ref |

import torch
import torch.nn as nn
import torch.nn.functional as F

class LPIPS(nn.Module):
    def __init__(self, net='alex'):
        super().__init__()
        import torchvision.models as models
        if net == 'alex':
            backbone = models.alexnet(pretrained=True).features
            channels = [64, 192, 384, 256, 256]
        self.slice1 = nn.Sequential(*list(backbone.children())[:2])
        self.slice2 = nn.Sequential(*list(backbone.children())[2:5])
        self.slice3 = nn.Sequential(*list(backbone.children())[5:8])
        self.slice4 = nn.Sequential(*list(backbone.children())[8:10])
        self.slice5 = nn.Sequential(*list(backbone.children())[10:12])
        self.weights = nn.ParameterList([
            nn.Parameter(torch.ones(c) / c) for c in channels])

    def forward(self, x, y):
        x_slices = [self.slice1(x), self.slice2(x),
                     self.slice3(x), self.slice4(x), self.slice5(x)]
        y_slices = [self.slice1(y), self.slice2(y),
                     self.slice3(y), self.slice4(y), self.slice5(y)]
        diffs = [(xs - ys) ** 2 for xs, ys in zip(x_slices, y_slices)]
        lpips = sum(
            (d.mean(dim=[2, 3]) * w).sum(dim=1)
            for d, w in zip(diffs, self.weights))
        return lpips.mean()

class QualityPredictor(nn.Module):
    def __init__(self, backbone_channels=2048):
        super().__init__()
        self.pool = nn.AdaptiveAvgPool2d(1)
        self.head = nn.Sequential(
            nn.Linear(backbone_channels, 512),
            nn.ReLU(inplace=True), nn.Dropout(0.5),
            nn.Linear(512, 128), nn.ReLU(inplace=True),
            nn.Linear(128, 1),
        )

    def forward(self, features):
        pooled = self.pool(features).flatten(1)
        return self.head(pooled)

Research Insight: Image quality assessment has been transformed by vision-language models: CLIPIQA leverages CLIP's perceptual understanding to predict quality scores that correlate with human judgments. The key insight is that "quality" is not a single number but a multi-dimensional concept (sharpness, noise, color fidelity, compression artifacts). Recent methods predict quality attributes jointly, enabling diagnostic assessment rather than just scalar scores. For quality enhancement, blind image super-resolution methods adapt their processing based on estimated degradation, achieving optimal results without knowing the degradation kernel.