Scikit-Learn Metrics

Introduction

Scikit-Learn provides comprehensive metrics for evaluating classification, regression, and clustering performance.

Classification Metrics

from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
from sklearn.metrics import confusion_matrix, classification_report

y_true = [0, 1, 0, 1, 0, 1]
y_pred = [0, 1, 0, 0, 1, 1]

accuracy = accuracy_score(y_true, y_pred)
precision = precision_score(y_true, y_pred)
recall = recall_score(y_true, y_pred)
f1 = f1_score(y_true, y_pred)

cm = confusion_matrix(y_true, y_pred)
print(classification_report(y_true, y_pred))

Regression Metrics

from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
import numpy as np

y_true = [1, 2, 3, 4, 5]
y_pred = [1.1, 2.2, 2.9, 3.8, 5.2]

mse = mean_squared_error(y_true, y_pred)
rmse = np.sqrt(mse)
mae = mean_absolute_error(y_true, y_pred)
r2 = r2_score(y_true, y_pred)

print(f"MSE: {mse}, RMSE: {rmse}, MAE: {mae}, R²: {r2}")

ROC and AUC

from sklearn.metrics import roc_curve, roc_auc_score
from sklearn.linear_model import LogisticRegression
import numpy as np

y_true = [0, 0, 1, 1]
y_scores = [0.1, 0.4, 0.35, 0.8]

fpr, tpr, thresholds = roc_curve(y_true, y_scores)
auc = roc_auc_score(y_true, y_scores)

print(f"AUC: {auc:.3f}")

Custom Scoring

from sklearn.metrics import make_scorer

def custom_metric(y_true, y_pred):
    return np.mean(np.abs(y_true - y_pred) < 0.5)

scorer = make_scorer(custom_metric)
scores = cross_val_score(model, X, y, cv=5, scoring=scorer)

Practice Problems

1. Calculate accuracy and F1 score
2. Generate confusion matrix
3. Compute RMSE and R²
4. Plot ROC curve
5. Create custom scoring function