Adversarial Robustness & Adversarial Training

1. The Adversarial Example Problem

Neural networks are vulnerable to adversarial examples — inputs crafted to cause misclassification with imperceptible perturbations:

1.1 Threat Models

Threat Model	Constraint	Attack
		FGSM, PGD
		C&W, PGD
		JSMA
Semantic	Meaning-preserving	Color, rotation

2. Adversarial Example in Input Space

<svg viewBox="0 0 800 400" xmlns="http://www.w3.org/2000/svg">
  <rect width="800" height="400" fill="#f8fafc"/>
  
  {/* Title */}
  <text x="400" y="30" text-anchor="middle" fill="#f8fafc" font-family="monospace" font-size="16" font-weight="bold">Adversarial Examples in Input Space</text>
  
  {/* Decision boundary */}
  <g transform="translate(50, 50)">
    <text x="350" y="0" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="12">Decision Boundary in Input Space</text>
    
    {/* Class regions */}
    <path d="M 0 0 L 700 0 L 700 300 L 0 300 Z" fill="#f8fafc" opacity="0.3"/>
    <path d="M 0 0 L 700 0 L 700 300 L 0 300 Z" fill="#3a1e5f" opacity="0.3" clip-path="url(#clip2)"/>
    
    {/* Decision boundary curve */}
    <path d="M 100 50 Q 350 100 500 200 Q 600 250 650 280" fill="none" stroke="#f59e0b" stroke-width="3" stroke-dasharray="10,5"/>
    <text x="550" y="180" fill="#fbbf24" font-family="monospace" font-size="10">Decision Boundary</text>
    
    {/* Original point */}
    <circle cx="300" cy="120" r="8" fill="#22c55e"/>
    <text x="300" y="100" text-anchor="middle" fill="#22c55e" font-family="monospace" font-size="10">x (correctly classified)</text>
    
    {/* Adversarial perturbation */}
    <line x1="300" y1="120" x2="380" y2="160" stroke="#ec4899" stroke-width="2"/>
    <circle cx="380" cy="160" r="8" fill="#ec4899"/>
    <text x="380" y="150" text-anchor="middle" fill="#ec4899" font-family="monospace" font-size="10">x_adv (misclassified)</text>
    
    {/* Perturbation vector */}
    <g transform="translate(320, 130)">
      <path d="M 0 0 L 50 30" stroke="#f59e0b" stroke-width="2" marker-end="url(#dc5d_arrowYellow3)"/>
      <text x="25" y="15" text-anchor="middle" fill="#fbbf24" font-family="monospace" font-size="8">δ (perturbation)</text>
    </g>
    
    {/* L_p ball */}
    <circle cx="300" cy="120" r="40" fill="none" stroke="#3b82f6" stroke-width="2" stroke-dasharray="5,3"/>
    <text x="300" y="170" text-anchor="middle" fill="#60a5fa" font-family="monospace" font-size="9">L₂ ball (ε=0.3)</text>
    
    {/* Axes */}
    <line x1="0" y1="300" x2="700" y2="300" stroke="#64748b" stroke-width="1"/>
    <line x1="0" y1="0" x2="0" y2="300" stroke="#64748b" stroke-width="1"/>
    <text x="350" y="320" text-anchor="middle" fill="#64748b" font-family="monospace" font-size="10">Feature Dimension 1</text>
    <text x="-20" y="150" text-anchor="middle" fill="#64748b" font-family="monospace" font-size="10" transform="rotate(-90, -20, 150)">Feature Dimension 2</text>
    
    {/* Class labels */}
    <text x="150" y="50" fill="#3b82f6" font-family="monospace" font-size="14" font-weight="bold">Class 0</text>
    <text x="550" y="280" fill="#a855f7" font-family="monospace" font-size="14" font-weight="bold">Class 1</text>
  </g>
  
  {/* Attack types */}
  <g transform="translate(50, 360)">
    <text x="350" y="0" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="11">Common Attacks</text>
    
    <rect x="0" y="15" width="120" height="30" rx="5" fill="#f8fafc" stroke="#3b82f6"/>
    <text x="60" y="35" text-anchor="middle" fill="#60a5fa" font-family="monospace" font-size="9">FGSM</text>
    
    <rect x="140" y="15" width="120" height="30" rx="5" fill="#f8fafc" stroke="#a855f7"/>
    <text x="200" y="35" text-anchor="middle" fill="#c084fc" font-family="monospace" font-size="9">PGD</text>
    
    <rect x="280" y="15" width="120" height="30" rx="5" fill="#f8fafc" stroke="#22c55e"/>
    <text x="340" y="35" text-anchor="middle" fill="#22c55e" font-family="monospace" font-size="9">C&W</text>
    
    <rect x="420" y="15" width="120" height="30" rx="5" fill="#f8fafc" stroke="#f59e0b"/>
    <text x="480" y="35" text-anchor="middle" fill="#fbbf24" font-family="monospace" font-size="9">AutoAttack</text>
    
    <rect x="560" y="15" width="120" height="30" rx="5" fill="#f8fafc" stroke="#ec4899"/>
    <text x="620" y="35" text-anchor="middle" fill="#ec4899" font-family="monospace" font-size="9">Square</text>
  </g>
  
  <defs>
    <marker id="dc5d_arrowYellow3" markerWidth="10" markerHeight="7" refX="9" refY="3.5" orient="auto">
      <polygon points="0 0, 10 3.5, 0 7" fill="#fbbf24"/>
    </marker>
    <clipPath id="clip2">
      <path d="M 0 0 Q 350 100 500 200 Q 600 250 700 300 L 0 300 Z"/>
    </clipPath>
  </defs>
</svg>

3. Fast Gradient Sign Method (FGSM)

3.1 Original FGSM (Goodfellow et al., 2015)

The gradient is computed with respect to the input, not the parameters.

3.2 Targeted FGSM

3.3 Iterative FGSM (I-FGSM)

4. Projected Gradient Descent (PGD)

4.1 PGD Attack (Madry et al., 2018)

where is the projection onto the ball around .

4.2 PGD as First-Order Adversary

PGD is a universal first-order adversary: given access to gradients only, it finds strong adversarial examples.

Algorithm:

Architecture Diagram

Input: x, y, ε, α, K (iterations)
x⁰ = x + random init in Lp ball
for t = 1 to K:
    xᵗ = xᵗ⁻¹ + α · sign(∇x L(θ, xᵗ⁻¹, y))
    xᵗ = Project(xᵗ, x, ε)
return xᴷ

4.3 PGD-2 vs. PGD-20

Iterations	Strength	Compute
PGD-1	Weak	1× forward + backward
PGD-7	Moderate	7×
PGD-20	Strong	20×
PGD-100	Very strong	100×

5. Decision Boundary with Robust Training

<svg viewBox="0 0 800 400" xmlns="http://www.w3.org/2000/svg">
  <rect width="800" height="400" fill="#f8fafc"/>
  
  {/* Title */}
  <text x="400" y="30" text-anchor="middle" fill="#f8fafc" font-family="monospace" font-size="16" font-weight="bold">Standard vs. Robust Decision Boundary</text>
  
  {/* Standard Training */}
  <g transform="translate(50, 60)">
    <text x="175" y="0" text-anchor="middle" fill="#ec4899" font-family="monospace" font-size="12" font-weight="bold">Standard Training</text>
    
    {/* Decision boundary */}
    <path d="M 20 50 Q 100 100 200 150 Q 250 200 300 250" fill="none" stroke="#ec4899" stroke-width="2"/>
    
    {/* Class regions */}
    <text x="100" y="100" fill="#3b82f6" font-family="monospace" font-size="10">Class 0</text>
    <text x="250" y="200" fill="#a855f7" font-family="monospace" font-size="10">Class 1</text>
    
    {/* Adversarial examples */}
    <circle cx="180" cy="120" r="5" fill="#f59e0b"/>
    <circle cx="190" cy="130" r="5" fill="#f59e0b"/>
    <circle cx="200" cy="140" r="5" fill="#f59e0b"/>
    
    {/* Lp balls */}
    <circle cx="180" cy="120" r="20" fill="none" stroke="#f59e0b" stroke-width="1" stroke-dasharray="3,2"/>
    
    <text x="175" y="280" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">Thin margin → vulnerable</text>
  </g>
  
  {/* Robust Training */}
  <g transform="translate(450, 60)">
    <text x="175" y="0" text-anchor="middle" fill="#22c55e" font-family="monospace" font-size="12" font-weight="bold">Robust Training</text>
    
    {/* Decision boundary (wider margin) */}
    <path d="M 50 50 Q 175 150 175 150 Q 175 150 300 250" fill="none" stroke="#22c55e" stroke-width="3"/>
    
    {/* Wide margin area */}
    <path d="M 30 30 Q 175 130 175 130 Q 175 130 320 230" fill="none" stroke="#22c55e" stroke-width="1" stroke-dasharray="5,3"/>
    <path d="M 70 70 Q 175 170 175 170 Q 175 170 280 270" fill="none" stroke="#22c55e" stroke-width="1" stroke-dasharray="5,3"/>
    
    {/* Class regions */}
    <text x="80" y="100" fill="#3b82f6" font-family="monospace" font-size="10">Class 0</text>
    <text x="250" y="200" fill="#a855f7" font-family="monospace" font-size="10">Class 1</text>
    
    {/* Robust points */}
    <circle cx="100" cy="80" r="5" fill="#22c55e"/>
    <circle cx="120" cy="100" r="5" fill="#22c55e"/>
    <circle cx="250" cy="220" r="5" fill="#22c55e"/>
    
    {/* Margin annotation */}
    <line x1="100" y1="80" x2="250" y2="220" stroke="#f59e0b" stroke-width="1" stroke-dasharray="3,2"/>
    <text x="175" y="140" text-anchor="middle" fill="#fbbf24" font-family="monospace" font-size="9">Wide margin → robust</text>
    
    <text x="175" y="280" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">Smooth gradient → stable</text>
  </g>
  
  {/* Key differences */}
  <g transform="translate(50, 340)">
    <rect width="700" height="40" rx="5" fill="#f8fafc" stroke="#334155"/>
    <text x="350" y="25" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="10">Robust training encourages smooth gradients and wide margins, but trades off accuracy on clean data</text>
  </g>
</svg>

6. Adversarial Training

6.1 Min-Max Formulation

where is the perturbation set.

6.2 TRADES (Zhang et al., 2019)

TRade-off-off between Adversarial and Standard Accuracy:

The adversarial loss trades off:

Clean accuracy:
Robustness: between clean and adversarial predictions

6.3 TRADES Loss Decomposition

7. Certified Robustness

7.1 Definition

A model is certified robust if for all and :

7.2 Interval Bound Propagation (IBP)

For a network with Lipschitz constant :

where is the margin between the top two logits.

7.3 Randomized Smoothing (Cohen et al., 2019)

Given a base classifier and noise distribution :

Theorem: If for some class , then is certified robust within radius:

where is the inverse CDF of the standard normal and is the probability of the second most likely class.

7.4 Gaussian Smoothing Certification

For robustness:

8. TRADES and Adversarial Training

<svg viewBox="0 0 800 400" xmlns="http://www.w3.org/2000/svg">
  <rect width="800" height="400" fill="#f8fafc"/>
  
  {/* Title */}
  <text x="400" y="30" text-anchor="middle" fill="#f8fafc" font-family="monospace" font-size="16" font-weight="bold">Adversarial Training Pipeline</text>
  
  {/* Training loop */}
  <g transform="translate(50, 60)">
    {/* Data batch */}
    <rect x="0" y="0" width="100" height="80" rx="8" fill="#f8fafc" stroke="#3b82f6" stroke-width="2"/>
    <text x="50" y="25" text-anchor="middle" fill="#60a5fa" font-family="monospace" font-size="10" font-weight="bold">Data</text>
    <text x="50" y="45" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">(x, y)</text>
    <text x="50" y="65" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">~ D</text>
    
    {/* PGD Attack */}
    <rect x="140" y="0" width="120" height="80" rx="8" fill="#f8fafc" stroke="#ec4899" stroke-width="2"/>
    <text x="200" y="25" text-anchor="middle" fill="#ec4899" font-family="monospace" font-size="10" font-weight="bold">PGD Attack</text>
    <text x="200" y="45" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">x_adv = x + δ*</text>
    <text x="200" y="65" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">δ* = argmax L</text>
    
    {/* Model */}
    <rect x="300" y="0" width="100" height="80" rx="8" fill="#f8fafc" stroke="#a855f7" stroke-width="2"/>
    <text x="350" y="25" text-anchor="middle" fill="#c084fc" font-family="monospace" font-size="10" font-weight="bold">Model</text>
    <text x="350" y="45" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">fθ(x_adv)</text>
    <text x="350" y="65" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">fθ(x)</text>
    
    {/* Loss */}
    <rect x="440" y="0" width="120" height="80" rx="8" fill="#f8fafc" stroke="#f59e0b" stroke-width="2"/>
    <text x="500" y="25" text-anchor="middle" fill="#fbbf24" font-family="monospace" font-size="10" font-weight="bold">Loss</text>
    <text x="500" y="45" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">L_clean + β·L_adv</text>
    <text x="500" y="65" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">TRADES</text>
    
    {/* Update */}
    <rect x="600" y="0" width="100" height="80" rx="8" fill="#f8fafc" stroke="#22c55e" stroke-width="2"/>
    <text x="650" y="25" text-anchor="middle" fill="#22c55e" font-family="monospace" font-size="10" font-weight="bold">Update</text>
    <text x="650" y="45" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">θ = θ - α∇L</text>
    <text x="650" y="65" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="9">Adam</text>
    
    {/* Arrows */}
    <line x1="100" y1="40" x2="140" y2="40" stroke="#64748b" stroke-width="2" marker-end="url(#dc5d_arrowGray4)"/>
    <line x1="260" y1="40" x2="300" y2="40" stroke="#64748b" stroke-width="2" marker-end="url(#dc5d_arrowGray4)"/>
    <line x1="400" y1="40" x2="440" y2="40" stroke="#64748b" stroke-width="2" marker-end="url(#dc5d_arrowGray4)"/>
    <line x1="560" y1="40" x2="600" y2="40" stroke="#64748b" stroke-width="2" marker-end="url(#dc5d_arrowGray4)"/>
    
    {/* Feedback loop */}
    <path d="M 650 80 L 650 110 L 50 110 L 50 80" fill="none" stroke="#22c55e" stroke-width="2" marker-end="url(#dc5d_arrowGreen4)"/>
    <text x="350" y="105" text-anchor="middle" fill="#22c55e" font-family="monospace" font-size="9">Epoch loop</text>
  </g>
  
  {/* TRADES loss visualization */}
  <g transform="translate(50, 200)">
    <rect width="700" height="180" rx="8" fill="#f8fafc" stroke="#334155"/>
    <text x="350" y="25" text-anchor="middle" fill="#f8fafc" font-family="monospace" font-size="12" font-weight="bold">TRADES Loss Components</text>
    
    {/* Clean loss */}
    <rect x="30" y="50" width="200" height="110" rx="5" fill="#f8fafc" stroke="#3b82f6"/>
    <text x="130" y="75" text-anchor="middle" fill="#60a5fa" font-family="monospace" font-size="10">Clean Loss</text>
    <text x="130" y="100" text-anchor="middle" fill="#f8fafc" font-family="monospace" font-size="9">L_clean = CE(fθ(x), y)</text>
    <text x="130" y="120" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="8">Standard cross-entropy</text>
    <text x="130" y="140" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="8">Encourages clean accuracy</text>
    
    {/* Robustness loss */}
    <rect x="250" y="50" width="200" height="110" rx="5" fill="#f8fafc" stroke="#a855f7"/>
    <text x="350" y="75" text-anchor="middle" fill="#c084fc" font-family="monospace" font-size="10">Robustness Loss</text>
    <text x="350" y="120" text-anchor="middle" fill="#f8fafc" font-family="monospace" font-size="9">L_adv = max DK L</text>
    <text x="350" y="120" text-anchor="middle" fill="#f8fafc" font-family="monospace" font-size="9">    (fθ(x) ∥ fθ(x+δ))</text>
    <text x="350" y="140" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="8">KL divergence under attack</text>
    <text x="350" y="155" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="8">Encourages local smoothness</text>
    
    {/* Trade-off */}
    <rect x="470" y="50" width="200" height="110" rx="5" fill="#f8fafc" stroke="#f59e0b"/>
    <text x="570" y="75" text-anchor="middle" fill="#fbbf24" font-family="monospace" font-size="10">Trade-off</text>
    <text x="570" y="100" text-anchor="middle" fill="#f8fafc" font-family="monospace" font-size="9">L_total = L_clean + β·L_adv</text>
    <text x="570" y="120" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="8">β controls accuracy-robustness</text>
    <text x="570" y="140" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="8">β=1.0 (standard TRADES)</text>
    <text x="570" y="155" text-anchor="middle" fill="#94a3b8" font-family="monospace" font-size="8">Higher β → more robust</text>
  </g>
  
  <defs>
    <marker id="dc5d_arrowGray4" markerWidth="10" markerHeight="7" refX="9" refY="3.5" orient="auto">
      <polygon points="0 0, 10 3.5, 0 7" fill="#64748b"/>
    </marker>
    <marker id="dc5d_arrowGreen4" markerWidth="10" markerHeight="7" refX="9" refY="3.5" orient="auto">
      <polygon points="0 0, 10 3.5, 0 7" fill="#22c55e"/>
    </marker>
  </defs>
</svg>

9. Advanced Attacks

9.1 Carlini & Wagner (C&W) Attack

where is the logit output and is the confidence margin.

9.2 AutoAttack (Croce & Hein, 2020)

Ensemble of attacks:

APGD-CE (CE loss)
APGD-DLR (difference of logits ratio)
FAB attack
Square attack (black-box)

9.3 Expectation over Transformation (EoT)

For robustness against spatial transformations:

10. Certified Robustness Methods

10.1 Randomized Smoothing

Certification pipeline:

Train base classifier with Gaussian noise
At test time, query times with noise
Compute confidence interval for
Certified radius:

10.2 Consistency Training

where .

10.3 Interval Bound Propagation (IBP)

For each layer :

The certified radius is:

where is the Lipschitz constant.

11. Accuracy-Robustness Trade-off

There is a fundamental trade-off between clean accuracy and adversarial robustness:

This trade-off can be mitigated by:

Larger models: More capacity for robust features
Better training: Early stopping, data augmentation
Ensemble methods: Combine multiple robust models
Feature alignment: Learn invariant representations

Understanding this trade-off is crucial for deploying robust models in safety-critical applications.

Adversarial Robustness & Adversarial Training

Adversarial Robustness & Adversarial Training

1. The Adversarial Example Problem

1.1 Threat Models

2. Adversarial Example in Input Space

3. Fast Gradient Sign Method (FGSM)

3.1 Original FGSM (Goodfellow et al., 2015)

3.2 Targeted FGSM

3.3 Iterative FGSM (I-FGSM)

4. Projected Gradient Descent (PGD)

4.1 PGD Attack (Madry et al., 2018)

4.2 PGD as First-Order Adversary

4.3 PGD-2 vs. PGD-20

5. Decision Boundary with Robust Training

6. Adversarial Training

6.1 Min-Max Formulation

6.2 TRADES (Zhang et al., 2019)

6.3 TRADES Loss Decomposition

7. Certified Robustness

7.1 Definition

7.2 Interval Bound Propagation (IBP)

7.3 Randomized Smoothing (Cohen et al., 2019)

7.4 Gaussian Smoothing Certification

8. TRADES and Adversarial Training

9. Advanced Attacks

9.1 Carlini & Wagner (C&W) Attack

9.2 AutoAttack (Croce & Hein, 2020)

9.3 Expectation over Transformation (EoT)

10. Certified Robustness Methods

10.1 Randomized Smoothing

10.2 Consistency Training

10.3 Interval Bound Propagation (IBP)

11. Accuracy-Robustness Trade-off

Need Expert AI/ML Premium Help?