Analysis of Observer Performance Based on Probing Patterns in an Interactive CAD System for Mammographic Mass Detection

PURPOSE In an interactive CAD system for reading mammograms users may probe regions for CAD information to improve decision making. Probed locations reveal information about the detection process. The purpose of this study is to assess to what extent missed lesions in an observer study were due to perception rather than interpretation failure. METHOD AND MATERIALS A mammographic workstation was developed in which readers can use CAD interactively. Instead of displaying all CAD findings as prompts readers can probe image regions for the presence of CAD information using a computer mouse. In an observer study we investigated the effect of this system on detection performance. Seven readers participated in the study, of which two were certified mammographers and five were non-radiologists with mammogram reading skills. The radiologists each read 60 cases including 20 with cancer, the non-radiologists read 120 cases including 40 cancer cases. All cancers selected were missed at the original screening and were retrospectively identified as visible. Cases with only microcalcifications were excluded. Mammograms were digitized from film and cases had up to 8 views when priors were available. Mammograms were read with and without CAD in two different sessions using a counter-balanced study design. The CAD system used was the ImageChecker V8.0 (R2/Hologic) and CAD was available in current and prior mammograms. From logfiles of the reading sessions we could determine locations that were probed during the CAD assisted sessions. Probed locations were correlated with true positive and false negative decisions. RESULTS Readers probed locations for CAD information 13.6 times per case on average, (9.9 for the radiologists). On average readers reported 68.8 % of the cancers while 80.8% of them were probed. Performance of the non-radiologists was similar to that of the radiologists. At a false positive recall rate of 10% the mean correct localisation fraction in the CAD assisted sessions was 46%. CONCLUSION Results confirm that most missed cancers are due to interpretation failure. On average more than 80% of the cancers were probed for CAD results while only 46% could be detected at a false positive recall rate of 10%. CLINICAL RELEVANCE/APPLICATION For further development of CAD it is needed to gain more understanding of perceptual problems in radiology. The proposed approach may serve as an alternative for eye-tracking.