Supervised Learning
Finding the Optimal Boundary — Maximum Margin Classification
SVM finds the hyperplane that maximizes the margin between classes. It is theoretically elegant and powerful in high-dimensional spaces.
- Maximum Margin — The widest possible gap between classes
- Kernel Trick — Nonlinear classification without explicit transformation
- Support Vectors — The critical points that define the decision boundary
"The art of discovery consists of seeing what everyone has seen and thinking what nobody has thought."
Support Vector Machines — Complete Guide
SVM finds the optimal hyperplane that maximizes the margin between classes. It is one of the most theoretically elegant ML algorithms.
Maximum Margin Classifier
Soft Margin SVM
The Kernel Trick
Kernel Decision Boundaries
Python Implementation
from sklearn.svm import SVC
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
# Linear SVM
pipe_linear = Pipeline([
('scaler', StandardScaler()),
('svm', SVC(kernel='linear', C=1.0))
])
pipe_linear.fit(X_train, y_train)
print(f"Linear: {pipe_linear.score(X_test, y_test):.3f}")
# RBF SVM (default)
pipe_rbf = Pipeline([
('scaler', StandardScaler()),
('svm', SVC(kernel='rbf', C=1.0, gamma='scale'))
])
pipe_rbf.fit(X_train, y_train)
print(f"RBF: {pipe_rbf.score(X_test, y_test):.3f}")
Key Takeaways
What to Learn Next
-> Logistic Regression Classification with probability — from linear to sigmoid.
-> Naive Bayes Bayes' theorem in action — fast, simple, surprisingly powerful.
-> Dimensionality Reduction Reduce features while preserving information with PCA and t-SNE.