AUC Confidence Interval

Basic Concepts

For large samples, AUC (area under the curve for a ROC curve) is approximately normally distributed, and so a 1-α confidence interval for AUC may be calculated as described in Confidence Interval for Sampling Distributions.

The confidence interval is equal to AUC  ± se · z_crit where z_crit is the two-tailed critical value of the standard normal distribution, as calculated in Excel by =NORM.S.INV(1-α/2) and

where n₁ and n₂ are the sizes of the two samples and

Example

Example 1: Find the 95% confidence for the AUC from Example 1 of Classification Table.

From Figure 1 of ROC Curve, we see that n₁ = 527, n₂ = 279, and AUC = .889515. The 95% confidence interval of AUC is (.867767, .911263), as shown in Figure 1.

Figure 1 – AUC 95% Confidence Interval

Worksheet Functions

Real Statistics Functions: The Real Statistics Resource Pack contains the following functions:

AUC_LOWER(auc, n1, n2, α) = the lower limit of the 1-α  confidence interval for the area under the curve = auc for samples of size n1 and n2

AUC_UPPER(auc, n1, n2, α) = the upper limit of the 1-α confidence interval for the area under the curve = auc for samples of size n1 and n2

If the α argument is omitted it defaults to .05.

For Example 1, we see that =AUC_LOWER(B5, B3, B4) calculates the value shown in cell B12 and =AUC_UPPER(B5, B3, B4) calculates the value shown in cell B13.

Examples Workbook

Click here to download the Excel workbook with the examples described on this webpage.

References

Hanley, J. A., McNeil, B. J. (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve
https://pubmed.ncbi.nlm.nih.gov/7063747/

Hintze, J. L. (2008) ROC Curves. NCSS
https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/ROC_Curves-Old_Version.pdf

Hintze, J. L. (2022) One ROC curve and cutoff analysis. NCSS
https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/One_ROC_Curve_and_Cutoff_Analysis.pdf

IBM (2011) ROC algorithms IBM SPSS Statistics 20 Algorithms
http://www.sussex.ac.uk/its/pdfs/SPSS_Algorithms_20.pdf