Omnidirectional Information Gathering for Knowledge Transfer-based
  Audio-Visual Navigation
Jinyu Chen Wenguan Wang Si Liu Hongsheng Li Yi Yang
Audio-visual navigation is an audio-targeted wayfinding task where a robot
agent is entailed to travel a never-before-seen 3D environment towards the
sounding source. In this article, we present ORAN, an omnidirectional
audio-visual navigator based on cross-task navigation skill transfer. In
particular, ORAN sharpens its two basic abilities for a such challenging task,
namely wayfinding and audio-visual information gathering. First, ORAN is
trained with a confidence-aware cross-task policy distillation (CCPD) strategy.
CCPD transfers the fundamental, point-to-point wayfinding skill that is well
trained on the large-scale PointGoal task to ORAN, so as to help ORAN to better
master audio-visual navigation with far fewer training samples. To improve the
efficiency of knowledge transfer and address the domain gap, CCPD is made to be
adaptive to the decision confidence of the teacher policy. Second, ORAN is
equipped with an omnidirectional information gathering (OIG) mechanism, i.e.,
gleaning visual-acoustic observations from different directions before
decision-making. As a result, ORAN yields more robust navigation behaviour.
Taking CCPD and OIG together, ORAN significantly outperforms previous
competitors. After the model ensemble, we got 1st in Soundspaces Challenge
2022, improving SPL and SR by 53% and 35% relatively.