Single Image Super-Resolution and GAN-Based Upscaling

Module: Computer Vision | Difficulty: Premium

Ill-Posed Problem

where is blur kernel, is downsampling, is noise.

Pixel Loss (PSNR)

Perceptual Loss (VGG)

where is the -th layer feature of VGG.

GAN Loss (Relativistic)

import torch
import torch.nn as nn

class ResidualDenseBlock(nn.Module):
    def __init__(self, channels=64, growth=32, num_layers=5):
        super().__init__()
        self.layers = nn.ModuleList()
        for i in range(num_layers):
            in_c = channels + i * growth
            self.layers.append(nn.Conv2d(in_c, growth, 3, 1, 1, bias=True))
        self.conv = nn.Conv2d(channels + num_layers * growth, channels, 1)
        self.relu = nn.LeakyReLU(0.2, inplace=True)

    def forward(self, x):
        features = [x]
        for layer in self.layers:
            out = layer(torch.cat(features, dim=1))
            features.append(self.relu(out))
        return self.conv(torch.cat(features, dim=1)) + x

class SRResNet(nn.Module):
    def __init__(self, scale=4, channels=64, num_blocks=16):
        super().__init__()
        self.head = nn.Conv2d(3, channels, 3, 1, 1)
        self.body = nn.Sequential(
            *[ResidualDenseBlock(channels) for _ in range(num_blocks)])
        self.body_conv = nn.Conv2d(channels, channels, 3, 1, 1)
        self.upsample = nn.Sequential(
            nn.Conv2d(channels, channels * scale * scale, 3, 1, 1),
            nn.PixelShuffle(scale),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Conv2d(channels, 3, 3, 1, 1),
        )

    def forward(self, x):
        head = self.head(x)
        body = self.body_conv(self.body(head)) + head
        return self.upsample(body)

Research Insight: Real-world super-resolution is fundamentally different from bicubic downsampling benchmarks. Real-ESRGAN introduced a second-order degradation model that simulates real camera artifacts (blur, noise, JPEG, resize). The key insight is that GAN-based methods achieve perceptually superior results but lower PSNR than MSE-optimized models, highlighting the misalignment between pixel-wise metrics and human perception. Diffusion-based SR methods (SR3, DiffBIR) now achieve state-of-the-art perceptual quality.