Quantitative

RMSE

The square root of MSE. \[ MSE=\frac{1}{n}\sum^n_{i=1}(y_i-y_i^{'})^2 \]

library(caret)
## Loading required package: lattice
## Loading required package: ggplot2
observed <- c(52, 19, 98)
predicted <- c(53, 30, 31)
RMSE(predicted, observed)
## [1] 39.20459

Coefficient of determination

R\(^2\):

  1. A correlation measurement.
  2. As the proportion of the variance explained by the model.
  3. Dependent on the variation in the outcome.
R2(predicted, observed)
## [1] 0.003160913

Rank correlation

Rank correlation takes the ranks of the observed outcome values and evaluates how close these are to ranks of the model predictions.

cor(predicted, observed)
## [1] -0.056222
cor(predicted, observed, method = "spearman")
## [1] 0.5

Variance-Bias Trande-off

To mitigating problem of collinearity by increasing model bias to reduce model variance.

E[MSE]=\(\sigma^2\)+(Model Bias)\(^2\)+Model Variance

Squared model bias reflects how closely the model can echo real relationship between predictors and the outcome. Simple models tend to under-fit and thus have high bias.

Variance increases with models complexity acommpanied with over-fitting or due to highly correlated predictors.