Right Place, Right Time! Towards ObjectNav for Non-Stationary Goals
Vishnu Sashank Dorbala Bhrij Patel Amrit Singh Bedi Dinesh Manocha
We present a novel approach to tackle the ObjectNav task for non-stationary
and potentially occluded targets in an indoor environment. We refer to this
task Portable ObjectNav (or P-ObjectNav), and in this work, present its
formulation, feasibility, and a navigation benchmark using a novel
memory-enhanced LLM-based policy. In contrast to ObjNav where target object
locations are fixed for each episode, P-ObjectNav tackles the challenging case
where the target objects move during the episode. This adds a layer of
time-sensitivity to navigation, and is particularly relevant in scenarios where
the agent needs to find portable targets (e.g. misplaced wallets) in
human-centric environments. The agent needs to estimate not just the correct
location of the target, but also the time at which the target is at that
location for visual grounding -- raising the question about the feasibility of
the task. We address this concern by inferring results on two cases for object
placement: one where the objects placed follow a routine or a path, and the
other where they are placed at random. We dynamize Matterport3D for these
experiments, and modify PPO and LLM-based navigation policies for evaluation.
Using PPO, we observe that agent performance in the random case stagnates,
while the agent in the routine-following environment continues to improve,
allowing us to infer that P-ObjectNav is solvable in environments with
routine-following object placement. Using memory-enhancement on an LLM-based
policy, we set a benchmark for P-ObjectNav. Our memory-enhanced agent
significantly outperforms their non-memory-based counterparts across object
placement scenarios by 71.76% and 74.68% on average when measured by Success
Rate (SR) and Success Rate weighted by Path Length (SRPL), showing the
influence of memory on improving P-ObjectNav performance. Our code and dataset
will be made publicly available.