ARIMA Models — Complete Guide

ARIMA (Autoregressive Integrated Moving Average) models combine autoregression, differencing, and moving average components to model and forecast time series data.

Component Models

AR(p) — Autoregressive

MA(q) — Moving Average

ARMA(p,q)

Model Identification Steps

| Step | Action |

|------|--------|

| 1 | Plot the series — look for trend, seasonality |

| 2 | Test stationarity — ADF and KPSS tests |

| 3 | Difference if needed — determine d |

| 4 | Examine ACF/PACF — identify p and q |

| 5 | Estimate model parameters |

| 6 | Diagnostics — check residuals |

| 7 | Forecast and evaluate |

Information Criteria

Lower AIC/BIC indicates a better model. BIC penalizes complexity more heavily.

Residual Diagnostics

After fitting, check that residuals are white noise:

1. Ljung-Box test: No autocorrelation in residuals

2. Normality test: Residuals approximately normal

3. Plot: No patterns in residuals vs. fitted values

Python Implementation

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

from statsmodels.tsa.arima.model import ARIMA

from statsmodels.tsa.stattools import adfuller

from statsmodels.graphics.tsaplots import plot_acf, plot_pacf



np.random.seed(42)



# Simulate AR(1) process

n = 300

y = np.zeros(n)

for t in range(1, n):

    y[t] = 0.7 * y[t-1] + np.random.randn()



# Stationarity test

adf = adfuller(y)

print(f"ADF p-value: {adf[1]:.4f}")



# Fit ARIMA(1,0,0) = AR(1)

model = ARIMA(y, order=(1, 0, 0))

results = model.fit()

print(results.summary())



# Diagnostics

print(f"\nLjung-Box p-value: {results.test_serial_correlation('ljungbox', lags=[10])[0]['lb_pvalue'].values[0]:.4f}")



# Forecast

forecast = results.forecast(steps=10)

print(f"\n10-step forecast: {forecast[:5].round(3)}")

Worked Example

Forecasting

Forecast accuracy is measured by:

| Metric | Formula | Interpretation |

|--------|---------|---------------|

| MAE | | Average absolute error |

| RMSE | | Penalizes large errors |

| MAPE | | Percentage error |

Key Takeaways

Related Topics

• See ACF and PACF for identifying model orders

• See Stationarity for testing stationarity

• See Seasonal Decomposition for seasonal ARIMA (SARIMA)