FGPrompt: Fine-grained Goal Prompting for Image-goal Navigation
Xinyu Sun Peihao Chen Jugang Fan Thomas H. Li Jian Chen Mingkui Tan
Learning to navigate to an image-specified goal is an important but
challenging task for autonomous systems. The agent is required to reason the
goal location from where a picture is shot. Existing methods try to solve this
problem by learning a navigation policy, which captures semantic features of
the goal image and observation image independently and lastly fuses them for
predicting a sequence of navigation actions. However, these methods suffer from
two major limitations. 1) They may miss detailed information in the goal image,
and thus fail to reason the goal location. 2) More critically, it is hard to
focus on the goal-relevant regions in the observation image, because they
attempt to understand observation without goal conditioning. In this paper, we
aim to overcome these limitations by designing a Fine-grained Goal Prompting
(FGPrompt) method for image-goal navigation. In particular, we leverage
fine-grained and high-resolution feature maps in the goal image as prompts to
perform conditioned embedding, which preserves detailed information in the goal
image and guides the observation encoder to pay attention to goal-relevant
regions. Compared with existing methods on the image-goal navigation benchmark,
our method brings significant performance improvement on 3 benchmark datasets
(i.e., Gibson, MP3D, and HM3D). Especially on Gibson, we surpass the
state-of-the-art success rate by 8% with only 1/50 model size. Project page:
https://xinyusun.github.io/fgprompt-pages