On mitigating stability-plasticity dilemma in CLIP-guided image morphing
  via geodesic distillation loss
Yeongtak Oh Saehyung Lee Uiwon Hwang Sungroh Yoon
Large-scale language-vision pre-training models, such as CLIP, have achieved
remarkable text-guided image morphing results by leveraging several
unconditional generative models. However, existing CLIP-guided image morphing
methods encounter difficulties when morphing photorealistic images.
Specifically, existing guidance fails to provide detailed explanations of the
morphing regions within the image, leading to misguidance. In this paper, we
observed that such misguidance could be effectively mitigated by simply using a
proper regularization loss. Our approach comprises two key components: 1) a
geodesic cosine similarity loss that minimizes inter-modality features (i.e.,
image and text) on a projected subspace of CLIP space, and 2) a latent
regularization loss that minimizes intra-modality features (i.e., image and
image) on the image manifold. By replacing the na\"ive directional CLIP loss in
a drop-in replacement manner, our method achieves superior morphing results on
both images and videos for various benchmarks, including CLIP-inversion.