Defense of Gregory Schuit

Abstract

Recently, generative neural network models have shown an increasing ability to synthesize high-quality images, largely due to the emergence of Diffusion Models. This has significant potential for the application of AI in radiology to address issues such as data sparsity, privacy, and explainability. However, ensuring medical correctness is crucial for the reliable generation of synthetic medical images. While several studies have evaluated the quality of these images using offline metrics, key attributes from the radiologists’ perspective have not been thoroughly studied. We aim to address this gap by quantifying realism and conditional correctness on chest X-ray generation and understanding radiologists’ reasoning concerning specific image attributes across four anomalies

We generated a dataset composed of three groups of images: real, GAN-generated, and diffusion-generated. We asked three radiologists to perform two tasks using a web interface to evaluate these images. The first task was identifying the synthetic image between two images, with the other being real. Radiologists were asked to explain their answers by selecting one of four possible anomalies. If none of the options applied, they could use a free-text field. The second task concerned conditional correctness. It consisted of judging if an abnormality was correctly assigned to an image. The results were processed and statistically analyzed, while free-text answers were further discussed with the participants.

Our study revealed that while Diffusion Models are superior to GAN in the general domain, they still do not ensure medical correctness when generating chest X-rays in certain conditions. While most synthetic images were indistinguishable from real ones, certain features could reveal their artificial nature. Our findings suggest that more work is needed to ensure realism and medical correctness when using DMs for radiology.