Interactive Semantic Map Representation for Skill-based Visual Object
  Navigation
Tatiana Zemskova Aleksei Staroverov Kirill Muravyev Dmitry Yudin Aleksandr Panov
Visual object navigation using learning methods is one of the key tasks in
mobile robotics. This paper introduces a new representation of a scene semantic
map formed during the embodied agent interaction with the indoor environment.
It is based on a neural network method that adjusts the weights of the
segmentation model with backpropagation of the predicted fusion loss values
during inference on a regular (backward) or delayed (forward) image sequence.
We have implemented this representation into a full-fledged navigation approach
called SkillTron, which can select robot skills from end-to-end policies based
on reinforcement learning and classic map-based planning methods. The proposed
approach makes it possible to form both intermediate goals for robot
exploration and the final goal for object navigation. We conducted intensive
experiments with the proposed approach in the Habitat environment, which showed
a significant superiority in navigation quality metrics compared to
state-of-the-art approaches. The developed code and used custom datasets are
publicly available at github.com/AIRI-Institute/skill-fusion.