Optimal Transport-Guided Conditional Score-Based Diffusion Models
Xiang Gu Liwei Yang Jian Sun Zongben Xu
Conditional score-based diffusion model (SBDM) is for conditional generation
of target data with paired data as condition, and has achieved great success in
image translation. However, it requires the paired data as condition, and there
would be insufficient paired data provided in real-world applications. To
tackle the applications with partially paired or even unpaired dataset, we
propose a novel Optimal Transport-guided Conditional Score-based diffusion
model (OTCS) in this paper. We build the coupling relationship for the unpaired
or partially paired dataset based on $L_2$-regularized unsupervised or
semi-supervised optimal transport, respectively. Based on the coupling
relationship, we develop the objective for training the conditional score-based
model for unpaired or partially paired settings, which is based on a
reformulation and generalization of the conditional SBDM for paired setting.
With the estimated coupling relationship, we effectively train the conditional
score-based model by designing a ``resampling-by-compatibility'' strategy to
choose the sampled data with high compatibility as guidance. Extensive
experiments on unpaired super-resolution and semi-paired image-to-image
translation demonstrated the effectiveness of the proposed OTCS model. From the
viewpoint of optimal transport, OTCS provides an approach to transport data
across distributions, which is a challenge for OT on large-scale datasets. We
theoretically prove that OTCS realizes the data transport in OT with a
theoretical bound. Code is available at \url{https://github.com/XJTU-XGU/OTCS}.