Introduction
Model evaluation assesses how well a model performs. Different metrics are used for classification and regression.
Classification Metrics
library(caret)
# Confusion matrix
confusionMatrix(pred, actual)
# Accuracy
mean(pred == actual)
# Precision, Recall, F1
postResample(pred, actual)
Regression Metrics
# RMSE
sqrt(mean((pred - actual)^2))
# MAE
mean(abs(pred - actual))
# R-squared
cor(pred, actual)^2
Cross-Validation
# K-fold CV
trainControl(method = "cv", number = 10)
# Repeated CV
trainControl(method = "repeatedcv",
number = 10, repeats = 3)
ROC Curve
library(pROC)
# Plot ROC
roc(actual, probabilities)
# AUC
auc(roc(actual, probabilities))
Summary
Use appropriate metrics for your problem. Cross-validation gives reliable performance estimates.