Cross Domain Generative Augmentation: Domain Generalization with Latent
  Diffusion Models
Sobhan Hemati Mahdi Beitollahi Amir Hossein Estiri Bassel Al Omari Xi Chen Guojun Zhang
Despite the huge effort in developing novel regularizers for Domain
Generalization (DG), adding simple data augmentation to the vanilla ERM which
is a practical implementation of the Vicinal Risk Minimization principle (VRM)
\citep{chapelle2000vicinal} outperforms or stays competitive with many of the
proposed regularizers. The VRM reduces the estimation error in ERM by replacing
the point-wise kernel estimates with a more precise estimation of true data
distribution that reduces the gap between data points \textbf{within each
domain}. However, in the DG setting, the estimation error of true data
distribution by ERM is mainly caused by the distribution shift \textbf{between
domains} which cannot be fully addressed by simple data augmentation techniques
within each domain. Inspired by this limitation of VRM, we propose a novel data
augmentation named Cross Domain Generative Augmentation (CDGA) that replaces
the pointwise kernel estimates in ERM with new density estimates in the
\textbf{vicinity of domain pairs} so that the gap between domains is further
reduced. To this end, CDGA, which is built upon latent diffusion models (LDM),
generates synthetic images to fill the gap between all domains and as a result,
reduces the non-iidness. We show that CDGA outperforms SOTA DG methods under
the Domainbed benchmark. To explain the effectiveness of CDGA, we generate more
than 5 Million synthetic images and perform extensive ablation studies
including data scaling laws, distribution visualization, domain shift
quantification, adversarial robustness, and loss landscape analysis.