Artificial Intelligence and Radiologists at Prostate Cancer Detection in MRI: The PI-CAI Challenge

PURPOSE: The PI-CAI challenge aims to validate the diagnostic performance of artificial intelligence (AI) and radiologists at clinically significant prostate cancer (csPCa) detection/diagnosis in MRI, with histopathology and follow-up (>= 3 years) as reference. The study hypothesizes that state-of-the-art AI algorithms, trained using thousands of patient exams, are non-inferior to radiologists reading bpMRI. MATERIALS AND METHODS: This retrospective study tentatively includes 12,373 prostate MRI exams (11,193 patients) between 2012 and 2021, curated from three Dutch centers and one Norwegian center. All patient exams are of men suspected of harboring csPCa, without a history of treatment or prior positive histopathology (Gleason Grade Group >= 2) findings. Acquisitions were obtained using Siemens Healthineers or Philips Medical Systems-based 1.5T/3T MRI scanners. In total, 11,373 cases are available to develop and train AI solutions in the framework of a grand challenge. The remaining 1000 unseen cases are reserved for testing. A subset of 400 testing cases is used to facilitate a reader study with 63 radiologists (42 centers, 18 countries; 1-23 years of experience reading prostate MRI median: 9 years). Readers perform their assessments sequentially for each patient: first, using bpMRI sequences only; secondly, using full mpMRI sequences (in compliance with PI-RADS v2.1). Multi-reader multi-case (MRMC) analysis is used to compare the overall patient-level diagnostic performance of readers against that of the top five AI algorithms. Non-inferiority is tested with a significance margin of 0.05. Permutation tests are used to statistically compare AI and radiologists' performance at PI-RADS operating points. The PI-CAI challenge spans 7 months (May-November 2022) in duration. Key aspects of its study protocol have been established in conjunction with an international scientific advisory board of 16 experts in prostate AI, radiology, and urology --to unify and standardize present-day guidelines, and ensure meaningful validation of prostate-AI towards clinical translation. RESULTS: Preliminary results of the PI-CAI challenge will be presented. CLINICAL RELEVANCE: Prostate MRI assessments show high inter-reader variability (>50%), long reporting times, and strong dependence on expertise. A thorough comparison of AI with radiologists builds trust, allowing AI to help improve diagnostic accuracy and reduce workload.