A Sentence Speaks a Thousand Images: Domain Generalization through
  Distilling CLIP with Language Guidance
Zeyi Huang Andy Zhou Zijian Lin Mu Cai Haohan Wang Yong Jae Lee
Domain generalization studies the problem of training a model with samples
from several domains (or distributions) and then testing the model with samples
from a new, unseen domain. In this paper, we propose a novel approach for
domain generalization that leverages recent advances in large vision-language
models, specifically a CLIP teacher model, to train a smaller model that
generalizes to unseen domains. The key technical contribution is a new type of
regularization that requires the student's learned image representations to be
close to the teacher's learned text representations obtained from encoding the
corresponding text descriptions of images. We introduce two designs of the loss
function, absolute and relative distance, which provide specific guidance on
how the training process of the student model should be regularized. We
evaluate our proposed method, dubbed RISE (Regularized Invariance with Semantic
Embeddings), on various benchmark datasets and show that it outperforms several
state-of-the-art domain generalization methods. To our knowledge, our work is
the first to leverage knowledge distillation using a large vision-language
model for domain generalization. By incorporating text-based information, RISE
improves the generalization capability of machine learning models.