Project AIR Sets the Standard for Commercial AI Products Validation in a Groundbreaking Publication in Radiology

The past decade has witnessed an exponential surge in commercially available AI-based solutions in radiology, with over 400 FDA-approved medical imaging software applications. While non-diagnostic AI applications have seen widespread adoption, the integration of diagnostic AI solutions into clinical settings has been hampered by various challenges. These challenges include the specificity of use cases, compatibility issues, regulatory compliance, opaque pricing structures, and a lack of rigorous evidence on their performance.

In a groundbreaking publication in Radiology, titled "Comparison of Commercial AI Software Performance for Radiograph Lung Nodule Detection and Bone Age Prediction", Kicky van Leeuwen et al. introduce Project AIR, a standardized method for the independent validation of commercially available artificial intelligence (AI) products in the field of radiology. This pioneering method not only ensures generalizability but also enables direct comparisons between competing products and human readers.

The study, which focuses on two crucial use cases — bone age prediction on hand radiographs and lung nodule detection on chest radiographs —, demonstrates that AI algorithms, when rigorously tested, can perform on par with or outperform experienced radiologists in specific scenarios. This achievement represents a significant milestone in the ongoing efforts to enhance transparency in the deployment of AI-based diagnostic software.

As the research community eagerly awaits subsequent projects following the same roadmap for additional use cases, the study by Kicky van Leeuwen undoubtedly marks a significant leap toward enhancing transparency and facilitating the broader adoption of AI solutions in clinical practice.

We're happy to congratulate Kicky van Leeuwen for the publication of her work! If you're interested in learning more about it, you can read the full paper and the editorial by Radiology.