ROC Curve

I am going to use Ttanic data to show you how to get ROC curve.

Cutoff of p = some value In the following code, if you keep on changing the value of p you will get different plots. TN = True Negative FN = False Negative TP = True Positive FP = False Positive

p <- 0.5
t1$predsur <- ifelse(t1$predprob >= p, 1,0)
t1$cat <- ifelse(t1$predsur == 0 & t1$Survived == 0, "TN",
          ifelse(t1$predsur == 0 & t1$Survived == 1, "FN",
          ifelse(t1$predsur == 1 & t1$Survived == 0, "FP","TP")))
set.seed(12345)
ggplot(data = t1, aes(x = Survived, y = predprob, color = factor(cat)))+ 
geom_jitter(size = 2, alpha = 1/2) +
scale_colour_manual(values = c("red", "green","blue","purple")) +
geom_hline(yintercept = p) +
theme_bw()

What if there was a perfect classifier? I am going to cheat a little bit in this code. Rather than finding a perfect classifier (which is unlikely to exist), I’m simply going to color False Positive and False Negative points white.

set.seed(12345)
ggplot(data = t1, aes(x = Survived, y = predprob, color = factor(cat)))+ 
geom_jitter(size = 2, alpha = 1/2) +
scale_colour_manual(values = c("white", "white","blue","purple")) +
geom_hline(yintercept = p) +
theme_bw()

Plot ROC curve

plot(roc2) + abline(a = 0, b=1)

numeric(0)

Get the AUC

performance(roc1, measure = "auc")@y.values

[[1]]
[1] 0.8418274