Supervised Learning

Bayes' Theorem in Action — Fast, Simple, Surprisingly Powerful

Naive Bayes applies Bayes' theorem with a "naive" independence assumption. Despite its simplicity, it often outperforms more complex algorithms on text classification.

Bayes' Theorem — The probabilistic foundation of classification
Conditional Independence — The simplifying assumption that makes it tractable
Gaussian / Multinomial / Bernoulli — Three variants for different data types

"Simplicity is the ultimate sophistication." — Leonardo da Vinci

Naive Bayes — Complete Guide

Naive Bayes applies Bayes' theorem with the "naive" assumption that features are conditionally independent given the class. Despite this simplification, it works remarkably well in practice.

Bayes' Theorem

The Naive Independence Assumption

Variants

Gaussian Naive Bayes

Multinomial Naive Bayes

Bernoulli Naive Bayes

Text Classification Example

from sklearn.naive_bayes import MultinomialNB
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.model_selection import cross_val_score

emails = ["Win free money now", "Claim your prize", "Meeting tomorrow at 10",
          "Project update", "Buy discount pills", "Special offer just for you",
          "Team lunch Friday", "Quarterly report attached"]
labels = [1, 1, 0, 0, 1, 1, 0, 0]  # 1=spam, 0=not spam

vectorizer = CountVectorizer()
X = vectorizer.fit_transform(emails)

model = MultinomialNB(alpha=1.0)  # Laplace smoothing
model.fit(X, labels)

# Predict on new email
new_emails = ["Free prize waiting for you"]
X_new = vectorizer.transform(new_emails)
print(f"Prediction: {model.predict(X_new)[0]}")  # 1 (spam)
print(f"P(spam|words) = {model.predict_proba(X_new)[0][1]:.4f}")

When to Use Naive Bayes

Key Takeaways

What to Learn Next

-> Logistic Regression Classification with probability — from linear to sigmoid.

-> SVM Finding the optimal boundary — maximum margin classification.

-> NLP Fundamentals Natural language processing — tokenization, embeddings, and transformers.

Naive Bayes — Complete Guide for Classification