Performance evaluation of automatic chest radiograph reading for detection of tuberculosis (TB): a comparative study with clinical officers and certified readers on TB suspects in sub-Saharan Africa

Purpose: Digital chest radiography (CXR) is used in high burden countries for suspect screening, active case finding and in prevalence surveys for TB diagnosis. An observer study was conducted to compare performance of automatic software with that of clinical officers and certified expert readers. Methods and Materials: A dataset of 166 digital CXRs (2048AfaEUR?2048, 0.25 mm, Delft Imaging Systems, The Netherlands) was collected at Kanyama Clinic, Lusaka, Zambia. Sputum culture was used as reference. An observer study was conducted with four clinical officers who read x-rays in Kanyama Clinic, and with two readers certified to read CXRs according to CRRS standard (University of Cape Town, South Africa). A software system for detection of TB (CAD4TB-1.08, Diagnostic Image Analysis Group, The Netherlands) analyzed all the cases. Human readers and software scored all the images between 0-100. We report area under the Receiver Operating Characteristics curve(Az) with 95% confidence intervals and pairwise comparisons from bootstrap estimates. p<0.05 was considered significant. Results: The dataset contained 99 positive and 67 negative cases. Az for the software was 0.72(0.63-0.80). Az values of CRRS certified readers were 0.71(0.63-0.79) and 0.72(0.64-0.78). Mean Az for clinical officers was 0.70(0.65-0.76). There was no significant difference between any reader and the software, except for one clinical officer who performed significantly worse than automatic reading. Conclusion: Automatic computer reading has similar performance as clinical officers and certified readers. The software has potential of being used as a point-of-care decision tool, to diagnose TB or select subjects that should undergo further tests.