What Is Near?: Room Locality Learning for Enhanced Robot
  Vision-Language-Navigation in Indoor Living Environments
Muraleekrishna Gopinathan Jumana Abu-Khalaf David Suter Sidike Paheding Nathir A. Rawashdeh
Humans use their knowledge of common house layouts obtained from previous
experiences to predict nearby rooms while navigating in new environments. This
greatly helps them navigate previously unseen environments and locate their
target room. To provide layout prior knowledge to navigational agents based on
common human living spaces, we propose WIN (\textit{W}hat \textit{I}s
\textit{N}ear), a commonsense learning model for Vision Language Navigation
(VLN) tasks. VLN requires an agent to traverse indoor environments based on
descriptive navigational instructions. Unlike existing layout learning works,
WIN predicts the local neighborhood map based on prior knowledge of living
spaces and current observation, operating on an imagined global map of the
entire environment. The model infers neighborhood regions based on visual cues
of current observations, navigational history, and layout common sense. We show
that local-global planning based on locality knowledge and predicting the
indoor layout allows the agent to efficiently select the appropriate action.
Specifically, we devised a cross-modal transformer that utilizes this locality
prior for decision-making in addition to visual inputs and instructions.
Experimental results show that locality learning using WIN provides better
generalizability compared to classical VLN agents in unseen environments. Our
model performs favorably on standard VLN metrics, with Success Rate 68\% and
Success weighted by Path Length 63\% in unseen environments.