Supervised Learning

From Scatter Plots to Predictions — The Simplest ML Algorithm

Linear regression finds the best straight line through your data. It is fast, interpretable, and a powerful baseline for any regression problem.

Ordinary Least Squares — The closed-form solution for optimal parameters
Gradient Descent — The iterative optimization approach that scales
Evaluation Metrics — R², MSE, and MAE for measuring performance

"All models are wrong, but some are useful." — George Box

Linear Regression — Complete Guide

Linear regression is the simplest and most fundamental ML algorithm. It models the relationship between variables as a straight line.

Simple Linear Regression

Finding the Best Line

Ordinary Least Squares (OLS)

Gradient Descent

Cost Function Surface and Gradient Descent Path

Multiple Linear Regression

Evaluation Metrics

Assumptions

Polynomial Regression


from sklearn.preprocessing import PolynomialFeatures

poly = PolynomialFeatures(degree=2)

X_poly = poly.fit_transform(X)

model = LinearRegression().fit(X_poly, y)

Key Takeaways

What to Learn Next

-> Logistic Regression

Classification with probability — from linear to sigmoid.

-> Regularization

Prevent overfitting with Ridge, Lasso, and Elastic Net.

-> Model Evaluation

How to know if your model actually works — beyond accuracy.

Linear Regression — Complete Guide with Math and Code