Performance of inexperienced and experienced observers in detection of active tuberculosis on digital chest radiographs with and without the use of computer-aided diagnosis

PURPOSE: Chest radiography is an important diagnostic test for the detection of tuberculosis (TB) but there are not enough experts to read chest radiographs (CXRs) in high burden countries. We compared the reading performance of inexperienced observers with and without the support of a computer-aided diagnosis (CAD) system with that of an experienced reader. METHOD AND MATERIALS: A set of 100 digital CXRs (Oldelca DR, Delft Imaging Systems, Veenendaal, The Netherlands) of TB suspects was collected from two sites in Sub-Saharan Africa. Sputum culture was used as the reference standard. All cases were scored on a scale of 0 to 100 for the presence of active TB by seven non-experts (undergraduate medical students) and one expert CRRS certified reader for reading CXRs for TB. Prior to reading, the non-experts received one hour of general instruction from a thoracic radiologist and one hour of case reading training with another CRRS certified reader. Cases were also processed by a CAD system (CAD4TB, version 1.08, Diagnostic Image Analysis Group, Nijmegen, The Netherlands). Scores of human readers and CAD were independently combined by averaging. Performance was evaluated as area under the ROC curve (Az), multi-reader-multi-case (MRMC) analysis was used to compare performance with and without CAD and pairwise comparisons were made with bootstrap estimation. p<0.05 was considered significant. RESULTS: The data set contained 56 negative and 44 positive cases. The expert reader scored Az = 0.84. The non-experts scored on average Az = 0.80, range 0.69-0.86. CAD standalone scored Az = 0.82. With CAD, all readers improved performance (Az = 0.85 for the expert, for non-experts average Az = 0.82, range 0.73-0.87). For four out of seven non-experts, the increase was significant and MRMC indicated an overall significant increase in performance for reading with CAD. CONCLUSION: Diagnostic performance of non-experts for detection of active TB on digital CXRs is good and similar to that of an expert. Support from CAD further improves performance. Digital chest radiography with reading of non-experts with little training can be used for active TB case finding at reasonable sensitivity with very high specificity, or as a first line test with high sensitivity and reasonable specificity. CLINICAL RELEVANCE/APPLICATION: Digital chest radiography and reading by inexperienced readers supported by CAD can be a fast, simple and low-cost point-of-care diagnostic for TB.