Augmenting medical image classifiers with synthetic data from latent
  diffusion models
Luke W. Sagers James A. Diao Luke Melas-Kyriazi Matthew Groh Pranav Rajpurkar Adewole S. Adamson Veronica Rotemberg Roxana Daneshjou Arjun K. Manrai
While hundreds of artificial intelligence (AI) algorithms are now approved or
cleared by the US Food and Drugs Administration (FDA), many studies have shown
inconsistent generalization or latent bias, particularly for underrepresented
populations. Some have proposed that generative AI could reduce the need for
real data, but its utility in model development remains unclear. Skin disease
serves as a useful case study in synthetic image generation due to the
diversity of disease appearance, particularly across the protected attribute of
skin tone. Here we show that latent diffusion models can scalably generate
images of skin disease and that augmenting model training with these data
improves performance in data-limited settings. These performance gains saturate
at synthetic-to-real image ratios above 10:1 and are substantially smaller than
the gains obtained from adding real images. As part of our analysis, we
generate and analyze a new dataset of 458,920 synthetic images produced using
several generation strategies. Our results suggest that synthetic data could
serve as a force-multiplier for model development, but the collection of
diverse real-world data remains the most important step to improve medical AI
algorithms.