Handling label noise through model confidence and uncertainty: application to chest radiograph classification

E. Calli, E. Sogancioglu, E.T. Scholten, K. Murphy and B. van Ginneken

in: Medical Imaging of Proceedings of the SPIE, 2019

Abstract

In this work we analyze the eect of label noise in training and test data when performing classication experi- ments on chest radiographs (CXRs) with modern deep learning architectures. We use ChestXRay14, the largest publicly available CXR dataset. We simulate situs inversus by horizontal ipping of the CXRs, allowing us to precisely control the amount of label noise. We also perform experiments in classifying emphysema using the ChestXRay14 provided labels that are known to be noisy. Our situs inversus experiments conrm results from the computer vision literature that deep learning architectures are relatively robust but not completely insensi- tive to label noise in the training data: without or with very low noise, classication results are near perfect; 16% and 32% training label noise only lead to a 1.5% and 4.6% drop in accuracy. We investigate two metrics that could be used to identify test samples that have an incorrect label: model condence and model uncertainty. We show, in an observer study with an experienced chest radiologist, that both measures are eective in identifying samples in ChestXRay14 that are erroneously labeled for the presence of emphysema.

A pdf file of this publication is available for personal use. Enter your e-mail address in the box below and press the button. You will receive an e-mail message with a link to the pdf file.