FRODO: An in-depth analysis of a system to reject outlier samples from a trained neural network.

An important limitation of state-of-the-art deep learning networks is that they do not recognize when their input is dissimilar to the data on which they were trained and proceed to produce outputs that will be unreliable or nonsensical. In this work, we describe FRODO (Free Rejection of Out-of-Distribution), a publicly available method that can be easily employed for any trained network to detect input data from a different distribution than is expected. FRODO uses the statistical distribution of intermediate layer outputs to define the expected in-distribution (ID) input image properties. New samples are judged based on the Mahalanobis distance (MD) of their layer outputs from the defined distribution. The method can be applied to any network, and we demonstrate the performance of FRODO in correctly rejecting OOD samples on three distinct architectures for classification, localization, and segmentation tasks in chest X-rays. A dataset of 21,576 X-ray images with 3,655 in-distribution samples is defined for testing. The remaining images are divided into four OOD categories of varying levels of difficulty, and performance at rejecting each type is evaluated using receiver operating characteristic (ROC) analysis. FRODO achieves areas under the ROC (AUC) of between 0.815 and 0.999 in distinguishing OOD samples of different types. This is shown to be comparable with the best-performing state-of-the-art method tested, with the substantial advantage that FRODO integrates seamlessly with any network and requires no extra model to be constructed and trained.