Artificial Intelligence and Radiologists at Prostate Cancer Detection in MRI: The PI-CAI Challenge

PURPOSE: The PI-CAI challenge aims to validate the diagnostic performance of artificial intelligence (AI) and radiologists at clinically significant prostate cancer (csPCa) detection in MRI, with histopathology and follow-up (>=3 years) as the reference standard. METHODS: This retrospective study includes 10,207 prostate MRI exams (9129 patients) curated from four European tertiary care centers between 2012-2021. All patients were men suspected of harboring prostate cancer, without a history of treatment or prior csPCa findings. Imaging was acquired using various 1.5 or 3T MRI scanners, equipped with surface coils. Algorithm developers worldwide were invited to develop AI models for detecting csPCa in bipara-metric MRI (bpMRI). For a given bpMRI exam, AI models were required to complete two tasks: localize all csPCa lesions (if any), and predict the case-level likelihood of csPCa diagnosis. To this end, AI models could use imaging data and several variables (e.g. patient age, PSA level, scanner model) to inform their predictions. Once devel-oped, these algorithms were independently tested using 1000 cases (including external data) in a fully-blinded setting. RESULTS: The PI-CAI study protocol was established in conjunction with 16 experts across prostate radiology, urology and AI. Between June-November 2022, >830 individuals (>50 countries) opted-in and >310 algorithm submissions were made. When trained on 1500 training cases, the top five most performant AI models reached 0.88+-0.01 AUROC in patient diagnosis, and 76.38+-0.74% sensitivity at 0.5 false positives per case in lesion detection. CONCLUSION: Preliminary findings indicate that the diagnostic performance of state-of-the-art AI models is comparable to that of radiologists reported in literature. LIMITATIONS: Radiology readings of the original data were used to guide biopsy planning, histopathology grading, and in turn, set the reference standard.