Diffusion Model with Clustering-based Conditioning for Food Image
  Generation
Yue Han Jiangpeng He Mridul Gupta Edward J. Delp Fengqing Zhu
Image-based dietary assessment serves as an efficient and accurate solution
for recording and analyzing nutrition intake using eating occasion images as
input. Deep learning-based techniques are commonly used to perform image
analysis such as food classification, segmentation, and portion size
estimation, which rely on large amounts of food images with annotations for
training. However, such data dependency poses significant barriers to
real-world applications, because acquiring a substantial, diverse, and balanced
set of food images can be challenging. One potential solution is to use
synthetic food images for data augmentation. Although existing work has
explored the use of generative adversarial networks (GAN) based structures for
generation, the quality of synthetic food images still remains subpar. In
addition, while diffusion-based generative models have shown promising results
for general image generation tasks, the generation of food images can be
challenging due to the substantial intra-class variance. In this paper, we
investigate the generation of synthetic food images based on the conditional
diffusion model and propose an effective clustering-based training framework,
named ClusDiff, for generating high-quality and representative food images. The
proposed method is evaluated on the Food-101 dataset and shows improved
performance when compared with existing image generation works. We also
demonstrate that the synthetic food images generated by ClusDiff can help
address the severe class imbalance issue in long-tailed food classification
using the VFN-LT dataset.