Tell, Don't Show!: Language Guidance Eases Transfer Across Domains in
  Images and Videos
Tarun Kalluri Bodhisattwa Prasad Majumder Manmohan Chandraker
We introduce LaGTran, a novel framework that utilizes readily available or
easily acquired text descriptions to guide robust transfer of discriminative
knowledge from labeled source to unlabeled target data with domain shifts.
While unsupervised adaptation methods have been established to address this
problem, they show limitations in handling challenging domain shifts due to
their exclusive operation within the pixel-space. Motivated by our observation
that semantically richer text modality has more favorable transfer properties,
we devise a transfer mechanism to use a source-trained text-classifier to
generate predictions on the target text descriptions, and utilize these
predictions as supervision for the corresponding images. Our approach driven by
language guidance is surprisingly easy and simple, yet significantly
outperforms all prior approaches on challenging datasets like GeoNet and
DomainNet, validating its extreme effectiveness. To further extend the scope of
our study beyond images, we introduce a new benchmark to study ego-exo transfer
in videos and find that our language-aided LaGTran yields significant gains in
this highly challenging and non-trivial transfer setting. Code, models, and
proposed datasets are publicly available at
https://tarun005.github.io/lagtran/.