Optimal Control for Neural Networks: Pontryagin's Principle

Module: Machine Learning | Difficulty: Advanced

Pontryagin's Maximum Principle

Hamiltonian

Optimal Control

Adjoint Equation

Connection to Neural ODEs

The loss gradient is computed via the adjoint method:

import torch
import torchdiffeq

class OptimalControlODE:
    def __init__(self, f, cost_fn):
        self.f = f; self.cost = cost_fn
    def hamiltonian(self, x, u, lam):
        return self.cost(x, u) + lam @ self.f(x, u)
    def adjoint_solve(self, x0, t_span):
        def adjoint(t, a):
            # a = lambda, compute -dH/dx
            x = self.x_trajectory(t)
            return -torch.autograd.grad(
                self.hamiltonian(x, self.u(t), a)[0], x, retain_graph=True)[0]
        a_T = torch.zeros_like(x0)
        return torchdiffeq.odeint(adjoint, a_T, t_span.flip(0))

Research Insight: The adjoint method computes gradients in constant memory regardless of the number of ODE steps. This is the key advantage of Neural ODEs over discrete ResNets, enabling training of infinitely deep networks.