Evaluation of a Computer-Aided Diagnosis system for Diabetic Retinopathy screening on public data

Purpose. To evaluate the performance of a comprehensivecomputer-aided diagnosis (CAD) system for Diabetic Retinopathy (DR) screening using a publicly available database of retinal images and compare its performance to that of human experts. Methods. A previously developed, comprehensive DR CAD system was applied to the 1200 digital color fundus photographs (non-mydriatic camera, single field) of 1200 eyes in the publicly available "Messidor" dataset. The ability of the system to distinguish normal images from those with DR was determined using Receiver Operator Characteristic (ROC) analysis. Two human experts also determined the presence of DR in each of the images. Results. The system achieved an area under the ROC curve of 0.876 for successfully distinguishing normal images from those with DR with a sensitivity of 92.2\% at a specificity of 50\%. This compares favorably with the two human experts who achieved sensitivities of 94.5\% and 91.2\% at specificity 50\%. Conclusions. This study shows, for the first time, the performance of a comprehensive DR screening system on an independent, publicly available dataset. The performance of the system on this dataset is comparable to that of human experts.