Logistic Regression
Overview
Logistic regression models the probability that a binary outcome given features . Unlike linear regression, it uses the sigmoid function to map the linear predictor to a probability between 0 and 1. The model is linear in the log-odds space: . Coefficients exponentiate to odds ratios (), providing intuitive effect size estimates. A classification threshold (typically 0.5) converts probabilities to binary predictions. The model is fitted via maximum likelihood estimation (MLE), not OLS.
Key Concepts
Classification Metrics
| Metric | Formula | Use Case |
|---|---|---|
| Accuracy | Balanced classes | |
| Precision | Cost of false positive high | |
| Recall | Cost of false negative high | |
| F1 Score | Balance precision and recall | |
| AUC-ROC | Area under ROC curve | Threshold-independent evaluation |
Odds Ratio Interpretation
| OR Value | Interpretation |
|---|---|
| OR = 1 | No association |
| OR > 1 | Positive association (increases odds) |
| OR < 1 | Negative association (decreases odds) |
| OR = 2 | Doubles the odds |
| OR = 0.5 | Halves the odds |
Quick Example
Key Takeaways
Deep Dive
For detailed explanations, worked examples, and Python implementations, explore the dedicated statistics lessons:
Logistic Regression
- Logistic Regression Statistics β Maximum likelihood estimation, coefficient interpretation, model diagnostics, and Python implementation
Odds Ratios
- Odds Ratios β Interpreting coefficients as odds ratios, confidence intervals, and practical examples
Related Topics
- Point Estimation β MLE theory underlying logistic regression estimation
- Simple Linear Regression β The continuous-outcome counterpart
- Chi-Square Test of Independence β Tests the same associations that logistic regression models
- Multiple Linear Regression β Extending to multiple predictors