Regression Models

Introduction

Regression algorithms for predicting continuous values.

Linear Regression

from sklearn.linear_model import LinearRegression, Ridge, Lasso

# Basic
model = LinearRegression()
model.fit(X_train, y_train)

# Ridge (L2 regularization)
ridge = Ridge(alpha=1.0)

# Lasso (L1 regularization)
lasso = Lasso(alpha=0.1)

Polynomial Regression

from sklearn.preprocessing import PolynomialFeatures

poly = PolynomialFeatures(degree=2)
X_poly = poly.fit_transform(X)

model = LinearRegression()
model.fit(X_poly, y)

Ensemble Methods

from sklearn.ensemble import RandomForestRegressor, GradientBoostingRegressor

# Random Forest
rf = RandomForestRegressor(n_estimators=100)
rf.fit(X_train, y_train)

# Gradient Boosting
gb = GradientBoostingRegressor(n_estimators=100, learning_rate=0.1)
gb.fit(X_train, y_train)

Practice Problems

1. Compare linear vs polynomial regression
2. Use regularization with Ridge/Lasso
3. Tune ensemble hyperparameters
4. Evaluate with cross-validation
5. Predict with ensemble models