Bridging Modality Gap for Visual Grounding with Effecitve Cross-modal
  Distillation
Jiaxi Wang Wenhui Hu Xueyang Liu Beihu Wu Yuting Qiu YingYing Cai
Visual grounding aims to align visual information of specific regions of
images with corresponding natural language expressions. Current visual
grounding methods leverage pre-trained visual and language backbones separately
to obtain visual features and linguistic features. Although these two types of
features are then fused via delicately designed networks, the heterogeneity of
the features makes them inapplicable for multi-modal reasoning. This problem
arises from the domain gap between the single-modal pre-training backbone used
in current visual grounding methods, which can hardly be overcome by the
traditional end-to-end training method. To alleviate this, our work proposes an
Empowering pre-trained model for Visual Grounding (EpmVG) framework, which
distills a multimodal pre-trained model to guide the visual grounding task.
EpmVG is based on a novel cross-modal distillation mechanism, which can
effectively introduce the consistency information of images and texts in the
pre-trained model, to reduce the domain gap existing in the backbone networks,
thereby improving the performance of the model in the visual grounding task.
Extensive experiments are carried out on five conventionally used datasets, and
results demonstrate that our method achieves better performance than
state-of-the-art methods.