ROC class notes

Le Kang

2024-03-18

Paired or unpaired comparisons for ROCs

Failure to take account of the fact that the same data set has been used to construct ROCs when comparing them is a not uncommon mistake.

Comparing summary statistics of two ROC curves

  • AUC
  • pAUC
  • the values of the curves at a particular point (e.g., the \(tp\) values for a particular value of the \(fp\); the \(tp\) values for a fixed value of the threshold \(t\); the misclassification rate for a fixed threshold \(t\)),
  • maximum vertical distance between the ROC curve and the chance diagonal

Comparing AUCs for two ROC curves

  • Unpaired case: use two sample Z test
  • Paired case: DeLong’s method 1 vs U-statistic method

What are the differences?

Comparing entire curves

  • Parametric approach
  • Nonparametric approach