Equivalence Testing

"Absence of evidence is not evidence of absence." — Altman & Bland, 2004

The Problem with Traditional Hypothesis Testing

In traditional testing:

• : (no difference)
• : (some difference)

Failing to reject does not prove equivalence. It merely means we lack evidence of a difference. This is fundamentally different from demonstrating that the treatments are equivalent.

Traditional vs Equivalence

Two One-Sided Tests (TOST)

Test Statistics

Equivalence is concluded when and , where .

Choosing the Equivalence Margin

Common Equivalence Margins

Bioequivalence

Log-Transformed Data

For bioequivalence, we work on the log scale:

Scaled Average Bioequivalence (SABE)

Power Analysis for Equivalence Testing

Non-Central t-Distribution Approach

Sample Size Formula (Two-Group Design)

For equal sample sizes and :

where is the critical value for the one-sided test and is the power quantile.

Relationship to Confidence Intervals

Decision rules:

Non-Inferiority Testing

This requires a historical evidence argument that the standard treatment has a known effect over placebo, and the margin preserves a fraction (typically 50%) of that effect.

Python Implementation

import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

# --- TOST for two independent groups ---
def tost_ind(x_treatment, x_reference, margin, alpha=0.05):
    """
    Two one-sided tests (TOST) for two independent groups.
    
    Parameters:
        x_treatment: array-like, treatment group data
        x_reference: array-like, reference group data
        margin: equivalence margin (Delta)
        alpha: significance level per test
    
    Returns:
        dict with test statistics, p-values, CI, and decision
    """
    n_t = len(x_treatment)
    n_r = len(x_reference)
    
    mean_t = np.mean(x_treatment)
    mean_r = np.mean(x_reference)
    diff = mean_t - mean_r
    
    # Pooled standard deviation
    var_t = np.var(x_treatment, ddof=1)
    var_r = np.var(x_reference, ddof=1)
    sp = np.sqrt(((n_t - 1) * var_t + (n_r - 1) * var_r) / (n_t + n_r - 2))
    se = sp * np.sqrt(1/n_t + 1/n_r)
    
    # TOST statistics
    df = n_t + n_r - 2
    t1 = (diff + margin) / se  # Test H01: diff <= -margin
    t2 = (diff - margin) / se  # Test H02: diff >= margin
    
    # P-values (one-sided)
    p1 = stats.t.sf(t1, df)   # P(T > t1) under H01
    p2 = stats.t.cdf(t2, df)  # P(T < t2) under H02
    
    # Combined p-value
    p_tost = max(p1, p2)
    
    # Confidence interval
    t_crit = stats.t.ppf(1 - alpha, df)
    ci_lower = diff - t_crit * se
    ci_upper = diff + t_crit * se
    
    # Decision
    reject = (p1 < alpha) and (p2 < alpha)
    
    return {
        'difference': diff,
        'se': se,
        't1': t1, 't2': t2,
        'p1': p1, 'p2': p2,
        'p_tost': p_tost,
        'ci_90': (ci_lower, ci_upper),
        'reject_equivalence': reject,
        'margin': margin,
        'df': df
    }

# Simulate bioequivalence study
np.random.seed(42)
# Log-transformed AUC values
n_subjects = 24
treatment = np.random.normal(6.8, 0.4, n_subjects)   # ln(AUC_T)
reference = np.random.normal(6.85, 0.4, n_subjects)   # ln(AUC_R)
delta = np.log(1.25)  # ~0.223 for 80-125% criteria

result = tost_ind(treatment, reference, margin=delta)
print("=== TOST Bioequivalence Test ===")
print(f"Mean difference (log scale): {result['difference']:.4f}")
print(f"Ratio (T/R): {np.exp(result['difference'])*100:.1f}%")
print(f"90% CI for ratio: ({np.exp(result['ci_90'][0])*100:.1f}%, "
      f"{np.exp(result['ci_90'][1])*100:.1f}%)")
print(f"T1 statistic: {result['t1']:.3f}, p1 = {result['p1']:.4f}")
print(f"T2 statistic: {result['t2']:.3f}, p2 = {result['p2']:.4f}")
print(f"Equivalence concluded: {result['reject_equivalence']}")

# --- Power curve for TOST ---
def tost_power(delta_true, margin, sigma, n, alpha=0.05):
    """Compute power of TOST for a given true difference."""
    se = sigma * np.sqrt(2 / n)
    df = 2 * n - 2
    t_crit = stats.t.ppf(1 - alpha, df)
    
    # Non-centrality parameters
    lambda1 = (delta_true + margin) / se
    lambda2 = (delta_true - margin) / se
    
    power1 = stats.nct.sf(t_crit, df, lambda1)
    power2 = stats.nct.cdf(-t_crit, df, lambda2)
    
    return power1 + power2 - 1  # Union of rejection regions

deltas = np.linspace(-0.3, 0.3, 100)
powers = [tost_power(d, delta, sigma=1.0, n=24) for d in deltas]

plt.figure(figsize=(10, 6))
plt.plot(deltas, powers, 'b-', linewidth=2)
plt.axvline(x=-delta, color='red', linestyle='--', label=f'Δ = ±{delta:.3f}')
plt.axvline(x=delta, color='red', linestyle='--')
plt.axhline(y=0.8, color='gray', linestyle=':', label='80% power')
plt.xlabel('True Mean Difference (δ)')
plt.ylabel('Power')
plt.title('Power Curve for TOST Equivalence Test (n=24 per group)')
plt.legend()
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.savefig('tost_power.png', dpi=150)
plt.show()

# --- Sample size calculation ---
def tost_sample_size(margin, sigma, delta=0, alpha=0.05, power=0.80):
    """Required sample size per group for TOST."""
    z_alpha = stats.norm.ppf(1 - alpha)
    z_beta = stats.norm.ppf(power)
    n = 2 * sigma**2 * (z_alpha + z_beta)**2 / (margin - abs(delta))**2
    return int(np.ceil(n))

n_req = tost_sample_size(delta, sigma=1.0, power=0.80)
print(f"\nRequired sample size per group: {n_req}")

Key Takeaways

Next Steps

• -> Hypothesis Testing
• -> Adaptive Trial Design
• -> Meta-Analysis