Mapping High-level Semantic Regions in Indoor Environments without
  Object Recognition
Roberto Bigazzi Lorenzo Baraldi Shreyas Kousik Rita Cucchiara Marco Pavone
Robots require a semantic understanding of their surroundings to operate in
an efficient and explainable way in human environments. In the literature,
there has been an extensive focus on object labeling and exhaustive scene graph
generation; less effort has been focused on the task of purely identifying and
mapping large semantic regions. The present work proposes a method for semantic
region mapping via embodied navigation in indoor environments, generating a
high-level representation of the knowledge of the agent. To enable region
identification, the method uses a vision-to-language model to provide scene
information for mapping. By projecting egocentric scene understanding into the
global frame, the proposed method generates a semantic map as a distribution
over possible region labels at each location. This mapping procedure is paired
with a trained navigation policy to enable autonomous map generation. The
proposed method significantly outperforms a variety of baselines, including an
object-based system and a pretrained scene classifier, in experiments in a
photorealistic simulator.