Assume we have a binary classifier that gives the probability of being a positive sample in the
[0.0, 1.0] range. Area Under the ROC Curve (AUC) quantitatively measures the accuracy of prediction made by such a classification model. Intuitively, what AUC does is to make sure if positive (i.e.,
label=1) samples in a validation set get higher probability of being positive than negative ones.
The AUC metric eventually gives a single value in
[0.0, 1.0]. When we have five test samples sorted by their prediction results as follows, we can see that the classifier put higher probability to all positive samples, #1, #2, and #4, than the others. We define the best scenario as an AUC of
|Test sample #||Probability of