Supervised Learning
Classification with Probability — From Linear to Sigmoid
Logistic regression transforms linear outputs into probabilities using the sigmoid function. It is the foundation of classification in machine learning.
- Sigmoid Function — Map any real number to a probability between 0 and 1
- Cross-Entropy Loss — The cost function that powers classification training
- Multiclass Extension — Softmax regression for multiple classes
"The goal is to turn data into information, and information into insight."
Logistic Regression — Complete Guide for Classification
Despite its name, logistic regression is a classification algorithm. It predicts the probability that an input belongs to a class.
From Linear to Logistic Regression
Cost Function
Decision Boundary
Gradient Derivation
Python Implementation
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
X, y = make_classification(n_samples=1000, n_features=10, n_informative=5, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
model = LogisticRegression(C=1.0, penalty='l2', solver='lbfgs')
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
y_prob = model.predict_proba(X_test)[:, 1]
from sklearn.metrics import accuracy_score, roc_auc_score
print(f"Accuracy: {accuracy_score(y_test, y_pred):.3f}")
print(f"AUC-ROC: {roc_auc_score(y_test, y_prob):.3f}")
Key Takeaways
What to Learn Next
-> Linear Regression From scatter plots to predictions — the simplest ML algorithm.
-> Naive Bayes Bayes' theorem in action — fast, simple, surprisingly powerful.
-> SVM Finding the optimal boundary — maximum margin classification.