Statistical Test for Generated Hypotheses by Diffusion Models
Teruyuki Katsuoka Tomohiro Shiraishi Daiki Miwa Vo Nguyen Le Duy Ichiro Takeuchi
The enhanced performance of AI has accelerated its integration into
scientific research. In particular, the use of generative AI to create
scientific hypotheses is promising and is increasingly being applied across
various fields. However, when employing AI-generated hypotheses for critical
decisions, such as medical diagnoses, verifying their reliability is crucial.
In this study, we consider a medical diagnostic task using generated images by
diffusion models, and propose a statistical test to quantify its reliability.
The basic idea behind the proposed statistical test is to employ a selective
inference framework, where we consider a statistical test conditional on the
fact that the generated images are produced by a trained diffusion model. Using
the proposed method, the statistical reliability of medical image diagnostic
results can be quantified in the form of a p-value, allowing for
decision-making with a controlled error rate. We show the theoretical validity
of the proposed statistical test and its effectiveness through numerical
experiments on synthetic and brain image datasets.