Image Harmonization

Module: Computer Vision | Difficulty: Advanced

Harmonization Problem

Given composite image :

Color Transfer

iS2S Loss

import torch
import torch.nn as nn

class HarmonizationNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Conv2d(7, 64, 3, padding=1), nn.ReLU(True),
            nn.Conv2d(64, 128, 3, 2, 1), nn.ReLU(True),
        )
        self.decoder = nn.Sequential(
            nn.ConvTranspose2d(128, 64, 4, 2, 1), nn.ReLU(True),
            nn.Conv2d(64, 3, 3, padding=1), nn.Tanh(),
        )
    
    def forward(self, composite, mask):
        inp = torch.cat([composite, mask], dim=1)
        return composite + mask * self.decoder(self.encoder(inp))

Key Takeaways

• Harmonization adjusts foreground to match background style
• Color transfer is a simple but effective baseline
• Deep methods handle complex illumination changes