A Classification Table (aka a confusion matrix) compares the predicted number of successes with the number of successes actually observed and similarly the predicted number of failures compared to the number actually observed.
The Receiver Operating Characteristic (ROC) Curve is a plot of values of the False Positive Rate (FPR) versus the True Positive Rate (TPR) for a specified cutoff value.
Topics
References
Wikipedia (2014) Receiver operating characteristic
https://en.wikipedia.org/wiki/Receiver_operating_characteristic
IBM SPSS (2016) Classification table
https://www.ibm.com/docs/en/spss-statistics/24.0.0?topic=model-classification-table
Hanley J. A., McNeil B. J. (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve
https://pubmed.ncbi.nlm.nih.gov/7063747/
Hintze, J. L. (2008) ROC curves. NCSS
https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/ROC_Curves-Old_Version.pdf
Hintze, J. L. (2022) One ROC curve and cutoff analysis. NCSS
https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/One_ROC_Curve_and_Cutoff_Analysis.pdf
IBM SPSS (2011) ROC algorithms IBM SPSS Statistics 20 Algorithms
http://www.sussex.ac.uk/its/pdfs/SPSS_Algorithms_20.pdf
Hi Charles,
Thank you for presenting auc confidence interval.
May I ask you how to use Excel to perform a test of the difference of two aucs from different rating system on the same sample.
Thanks
Hi Lily,
See https://ncss-wpengine.netdna-ssl.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Comparing_Two_ROC_Curves-Paired_Design.pdf
Charles
When I try to run the ROC analysis I get this advice from excel
Input Range must only have non-negative integer data entries; first non-valid entry is at cell XX
what was wrong?
Domenico,
Do you have non-negative entries? The values in the input range must be counts, which are non-negative entries.
Charles
Hi Charles,
Thanks a lot for the reply. I have re-installed the real statistics excel add in and it is working fine. But I am afraid I do not know much of statistics and hence unable to use it. I am basically working on relaibliity of diagnostic test. Can you help me out by plotting an ROC curve if I send you the excel file of observations?
I have calculated the sensetivity, specificity, PPV and NPV etc
Hello,
I am very happy to answer questions and help people to use the statistical analysis tools that are described on the Real Statistics website, but I don’t have the time or desire to perform these analyses. I am sorry.
Charles
Hi Charles,
Thanks a lot for the reply and thanks for all the help offered
I am studying diagnostic accuracy of cystoscopy to detect bladder cancer.
In this study cystoscopy images in three different light settings were recorded. These light settings are Light A, Light B and Light W. Total 45 images were used in the study. So there are 45 rows excluding the header row which contains unique code number of the observer.
These images were shown to observers and they were asked to reply whether they feel the said image is a cancer or not. They were given to tick one of the options: 1. Cancer yes, 2. Cancer no, 3. Can’t say
The responses from the observers were judged against histopathology report of the lesion in the image shown to the observer. The observers were not aware of this report.
My data is arranged as below
Column 1 is Histopathology report: cancer yes or cancer No (the observers did not have access to this report)
Column 2 denotes which light was used to record the image: Light A. Light B or Light W
Column 3 and the next 272 columns after that note the response of individual observer as to yes, no or Can’t say (total 273 observers participated in this study)
I want to draw the ROC curve for light A. B and W. I need to exclude “can’t say” observations. I have calculated sensitivity, specificity, PPV and NPV manually
How do I do this?