Multimodal Classification of Breast Masses in Mammography and MRI using Unimodal Feature Selection and Decision Fusion

In this work, a classifier combination approach for computer aided diagnosis (CADx) of breast mass lesions in mammography (MG) and magnetic resonance imaging (MRI) is investigated, using a database with 278 and 243 findings in MG resp. MRI including 98 multimodal (MM) lesion annotations. For each modality, feature selection was performed separately with linear Support Vector Machines (SVM). Using nonlinear SVMs, calibrated unimodal malignancy estimates were obtained and fused to a multimodal (MM) estimate by averaging. Evaluating the area under the receiver operating characteristic curve (AUC), feature selection raised AUC from 0.68, 0.69 and 0.72 for MG, MRI and MM to 0.76, 0.73 and 0.81 with a significant improvement for MM (P=0.018). Multimodal classification offered increased performance compared to MG and MRI (P=0.181 and P=0.087). In conclusion, unimodal feature selection significantly increased multimodal classification performance and can provide a useful tool for generating joint CADx scores in the multimodal setting.