DELAN: Dual-Level Alignment for Vision-and-Language Navigation by
  Cross-Modal Contrastive Learning
Mengfei Du Binhao Wu Jiwen Zhang Zhihao Fan Zejun Li Ruipu Luo Xuanjing Huang Zhongyu Wei
Vision-and-Language navigation (VLN) requires an agent to navigate in unseen
environment by following natural language instruction. For task completion, the
agent needs to align and integrate various navigation modalities, including
instruction, observation and navigation history. Existing works primarily
concentrate on cross-modal attention at the fusion stage to achieve this
objective. Nevertheless, modality features generated by disparate uni-encoders
reside in their own spaces, leading to a decline in the quality of cross-modal
fusion and decision. To address this problem, we propose a Dual-levEL AligNment
(DELAN) framework by cross-modal contrastive learning. This framework is
designed to align various navigation-related modalities before fusion, thereby
enhancing cross-modal interaction and action decision-making. Specifically, we
divide the pre-fusion alignment into dual levels: instruction-history level and
landmark-observation level according to their semantic correlations. We also
reconstruct a dual-level instruction for adaptation to the dual-level
alignment. As the training signals for pre-fusion alignment are extremely
limited, self-supervised contrastive learning strategies are employed to
enforce the matching between different modalities. Our approach seamlessly
integrates with the majority of existing models, resulting in improved
navigation performance on various VLN benchmarks, including R2R, R4R, RxR and
CVDN.