Automatic detection of hemorrhages on color fundus images using deep learning

Purpose: The presence of hemorrhages is one of the common signs of diabetic retinopathy (DRP), a vision threatening retinal disease affecting patient with diabetes. Automatic detection of hemorrhages is important to facilitate the timely detection of patients that would benefit from treatment to slow down or prevent disease progression to vision-threatening stages of DRP. We report an automatic system based on deep learning to automatically detect hemorrhages on color fundus images. Methods: Data was drawn from two public datasets (Kaggle and Messidor) by selecting images with sufficient quality for analysis, including a total of 4624 and 1102 images, respectively. The Kaggle set was split into a development (4048) and an evaluation set (576). The Messidor set was solely used as external evaluation set. A reference observer indicated presence of hemorrhages for all images and also marked their locations in the development set. Both evaluation sets were also scored for the presence of hemorrhages by two independent human experts. An automatic system based on deep learning, employing a convolutional neural network (CNN), was developed and used to identify images with hemorrhages. The CNN consisted of a layered architecture of 5 convolutional layers, a fully connected layer, and a final classification layer. The CNN used 41x41 pixel sized color patches as input in the first layer. In each convolutional layer, inputs were convolved with a set of small sized filters and the response maps were used as input in the next layer. In the last layer, an image score indicating the likelihood for the presence of hemorrhages was generated. Evaluation was performed by comparing system results and human expert annotations with the reference. Results: The reference observer marked 99 and 289 images as containing hemorrhages; and 477 and 813 as controls in the two test sets. The automatic system achieved areas (Az) under the receiver operating characteristics curve of 0.957 and 0.968 with sensitivity/specificity pairs of 0.889/0.912 and 0.931/0.888, whereas the human experts achieved sensitivity/specificity of 0.919/0.979 and 0.899/0.977 in the Kaggle test set; and 0.976/0.894 and 0.958/0.872 in the Messidor test set. Conclusions: An automatic system was developed for the detection of hemorrhages on color fundus images. The system approaches human expert performance and allows for a fast and reliable identification of patients with moderate to severe DRP in a screening setup.