Specialized Topics

Time Series — When Order Matters More Than Magnitude

Time series data is ordered by time — stock prices, weather, sales — and forecasting predicts future values based on historical patterns.

ARIMA — the classical statistical approach combining autoregression, differencing, and moving averages
Prophet — Facebook's tool that handles seasonality, holidays, and missing data automatically
LSTM Networks — deep learning models that capture complex temporal dependencies and nonlinear patterns

"The best way to predict the future is to study the past." — Robert Kiyosaki

Prerequisites

Before diving into time series analysis, you should be familiar with:

Statistics Basics — mean, variance, standard deviation, distributions
Linear Regression — understanding of trend modeling
Pandas — DataFrame manipulation and time indexing
Matplotlib — data visualization
Basic Probability — understanding of random processes
Python Programming — familiarity with NumPy and pandas

Learning Objectives

By the end of this tutorial, you will be able to:

Decompose time series into trend, seasonality, and residual components
Test for stationarity using the Augmented Dickey-Fuller test
Make a time series stationary through differencing and transformations
Implement ARIMA models for forecasting
Use Facebook Prophet for automatic seasonality detection
Build LSTM neural networks for complex temporal patterns
Perform proper time series cross-validation (walk-forward validation)
Engineer time-based features for machine learning models
Evaluate forecast accuracy with appropriate metrics
Handle common time series challenges like missing data and outliers

Time Series Components

Time Series Components Diagram

Stationarity

Stationarity Visualization

MathExample: Stationarity Testing

from statsmodels.tsa.stattools import adfuller
import numpy as np
import pandas as pd

# Generate non-stationary data (random walk)
np.random.seed(42)
n = 500
random_walk = np.cumsum(np.random.randn(n))

# ADF test
result = adfuller(random_walk)
print(f'ADF Statistic: {result[0]:.4f}')
print(f'p-value: {result[1]:.4f}')
print(f'Critical Values: {result[4]}')

# If p-value > 0.05, series is non-stationary
if result[1] > 0.05:
    print("Series is non-stationary, applying differencing...")
    diff_series = np.diff(random_walk)
    result_diff = adfuller(diff_series)
    print(f'After differencing - ADF: {result_diff[0]:.4f}, p: {result_diff[1]:.4f}')

ARIMA

ACF/PACF Diagram

MathExample: Complete ARIMA Workflow

import pandas as pd
import numpy as np
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
import matplotlib.pyplot as plt

# Generate synthetic time series
np.random.seed(42)
n = 200
trend = np.linspace(0, 10, n)
seasonal = 2 * np.sin(2 * np.pi * np.arange(n) / 50)
noise = np.random.randn(n) * 0.5
y = trend + seasonal + noise

# Split into train/test
train_size = int(len(y) * 0.8)
train, test = y[:train_size], y[train_size:]

# Fit ARIMA(1,1,1)
model = ARIMA(train, order=(1, 1, 1))
fitted = model.fit()

# Forecast
forecast = fitted.forecast(steps=len(test))

# Calculate error
from sklearn.metrics import mean_squared_error, mean_absolute_error
rmse = np.sqrt(mean_squared_error(test, forecast))
mae = mean_absolute_error(test, forecast)
print(f"RMSE: {rmse:.4f}")
print(f"MAE: {mae:.4f}")

# Plot
plt.figure(figsize=(12, 6))
plt.plot(range(len(train)), train, label='Train')
plt.plot(range(len(train), len(train) + len(test)), test, label='Test')
plt.plot(range(len(train), len(train) + len(forecast)), forecast, label='Forecast')
plt.legend()
plt.title('ARIMA Forecast')
plt.show()

Facebook Prophet

MathExample: Prophet with Holidays and Events

from prophet import Prophet
import pandas as pd

# Create dataframe with dates and values
df = pd.DataFrame({
    'ds': pd.date_range(start='2020-01-01', periods=1000, freq='D'),
    'y': np.random.randn(1000).cumsum() + 100
})

# Add custom holidays
holidays = pd.DataFrame({
    'holiday': 'black_friday',
    'ds': pd.to_datetime(['2020-11-27', '2021-11-26', '2022-11-25']),
    'lower_window': -1,
    'upper_window': 1,
})

model = Prophet(
    holidays=holidays,
    yearly_seasonality=True,
    weekly_seasonality=True,
    changepoint_prior_scale=0.05
)
model.fit(df)

future = model.make_future_dataframe(periods=365)
forecast = model.predict(future)

# Components plot
fig = model.plot_components(forecast)
plt.show()

LSTM for Time Series

MathNote: LSTM vs Traditional Methods

Key Formulas Reference

Essential Time Series Formulas

Formula	Description
`y'(t) = y(t) - y(t-1)`	First-order differencing
`AR(p): y(t) = c + Σ φᵢy(t-i) + ε(t)`	Autoregressive model
`MA(q): y(t) = c + ε(t) + Σ θⱼε(t-j)`	Moving average model
`RMSE = √(Σ(ŷ-y)²/n)`	Root mean squared error
`MAPE = (1/n)Σ	y-ŷ

Time Series Feature Engineering

import pandas as pd

def create_time_features(df):
    df = df.copy()
    
    # Lag features
    for lag in [1, 7, 14, 30]:
        df[f'lag_{lag}'] = df['value'].shift(lag)
    
    # Rolling statistics
    for window in [7, 14, 30]:
        df[f'rolling_mean_{window}'] = df['value'].rolling(window).mean()
        df[f'rolling_std_{window}'] = df['value'].rolling(window).std()
    
    # Calendar features
    df['day_of_week'] = df.index.dayofweek
    df['month'] = df.index.month
    df['quarter'] = df.index.quarter
    df['is_weekend'] = df.index.dayofweek >= 5
    
    return df.dropna()

Real-World Applications

1. Stock Price Forecasting

Predicting future stock prices using historical price data, volume, and technical indicators. ARIMA and LSTM are commonly used.

2. Demand Forecasting

Retailers predict product demand to optimize inventory. Prophet handles holiday effects and promotions well.

3. Weather Prediction

Meteorological forecasting uses time series models to predict temperature, rainfall, and extreme weather events.

4. Energy Consumption

Utility companies forecast electricity demand to optimize power generation and grid management.

5. Website Traffic

E-commerce sites predict traffic patterns for capacity planning and marketing campaign timing.

6. Healthcare Monitoring

Patient vital signs monitoring uses time series analysis for early detection of anomalies and deterioration.

Common Mistakes & How to Avoid Them

1. Using Random Train-Test Split

Time series must be split temporally. Random splitting causes data leakage where future data leaks into training.

2. Ignoring Stationarity

ARIMA assumes stationarity. Always test with ADF and apply differencing if needed.

3. Overfitting to Noise

Complex models like LSTM can memorize noise. Use simple baselines first (naive forecast, moving average).

4. Not Handling Missing Data

Time series often have gaps. Use interpolation or imputation methods designed for temporal data.

5. Ignoring Seasonality

Forgetting to account for daily, weekly, or yearly patterns leads to poor forecasts.

6. Evaluating on Training Data

Always evaluate on a held-out test set that comes after the training period.

Interview Questions

Q1: What is the difference between ARIMA and SARIMA?

A: SARIMA (Seasonal ARIMA) extends ARIMA by adding seasonal components (P, D, Q, s) to model periodic patterns. Use SARIMA when data shows clear seasonal patterns like yearly or weekly cycles.

Q2: How do you choose p, d, q for ARIMA?

A: Use ACF and PACF plots. PACF cuts off at lag p (AR order), ACF cuts off at lag q (MA order). For d, difference until stationary (ADF test p < 0.05). Auto-ARIMA can automate this.

Q3: When would you use Prophet over ARIMA?

A: Prophet handles missing data, outliers, multiple seasonalities, and holiday effects automatically. It's better for business time series with irregular patterns. ARIMA is better for stationary, purely statistical patterns.

Q4: What is walk-forward validation?

A: Instead of random CV splits, walk-forward validation trains on past data and tests on the next period, then slides forward. This respects temporal ordering and avoids data leakage.

Q5: How do LSTM networks handle time series differently?

A: LSTM can capture long-term dependencies and nonlinear patterns that linear models like ARIMA cannot. It learns complex patterns from raw sequences but requires more data and is less interpretable.

Q6: What are exogenous variables in time series?

A: External variables that influence the time series but aren't forecasted themselves. Examples: temperature affecting ice cream sales, holidays affecting retail traffic. ARIMAX and Prophet support exogenous variables.

Q7: How do you handle missing values in time series?

A: Methods include forward/backward fill, linear interpolation, seasonal decomposition, or model-based imputation. The choice depends on the pattern and amount of missingness.

Practice Exercise

Exercise: Forecast Store Sales

Objective: Build a forecasting model for daily store sales.

Dataset: Create synthetic data with:

2 years of daily sales data
Weekly seasonality (lower on Sundays)
Yearly seasonality (higher in December)
Upward trend
Random noise

Tasks:

Generate and visualize the time series
Test for stationarity and apply differencing if needed
Decompose the time series into components
Fit ARIMA model and forecast 30 days ahead
Fit Prophet model with weekly and yearly seasonality
Compare models using RMSE and MAE
Analyze residuals — are they white noise?

Bonus: Add holiday effects and compare how each model handles them.

Comparison Table

Time Series Methods Comparison

Feature	ARIMA	Prophet	LSTM
Assumptions	Stationarity	Additive components	None (non-linear)
Seasonality	Manual (SARIMA)	Automatic	Learned from data
Missing Data	Not handled	Handled automatically	Not handled
Interpretability	High	High	Low
Data Requirement	Medium	Medium	Large
Non-linear Patterns	No	Limited	Yes
Computational Cost	Low	Low	High

Key Takeaways

What to Learn Next

-> Linear Regression Understand the foundation for time series trend modeling and simple forecasting methods.

-> RNN and LSTM Apply recurrent neural networks to capture complex temporal patterns in sequential data.

-> NLP Fundamentals Explore text processing techniques that share tokenization and embedding concepts with time series.

-> Model Evaluation Learn time-series-specific validation strategies like walk-forward cross-validation.

-> Reinforcement Learning Extend sequential decision-making to agent-environment interaction problems.

-> Recommendation Systems Apply user-item interaction modeling which often involves temporal patterns.

Time Series Analysis and Forecasting Complete Guide