PreSight: Enhancing Autonomous Vehicle Perception with City-Scale NeRF
  Priors
Tianyuan Yuan Yucheng Mao Jiawei Yang Yicheng Liu Yue Wang Hang Zhao
Autonomous vehicles rely extensively on perception systems to navigate and
interpret their surroundings. Despite significant advancements in these systems
recently, challenges persist under conditions like occlusion, extreme lighting,
or in unfamiliar urban areas. Unlike these systems, humans do not solely depend
on immediate observations to perceive the environment. In navigating new
cities, humans gradually develop a preliminary mental map to supplement
real-time perception during subsequent visits. Inspired by this human approach,
we introduce a novel framework, Pre-Sight, that leverages past traversals to
construct static prior memories, enhancing online perception in later
navigations. Our method involves optimizing a city-scale neural radiance field
with data from previous journeys to generate neural priors. These priors, rich
in semantic and geometric details, are derived without manual annotations and
can seamlessly augment various state-of-the-art perception models, improving
their efficacy with minimal additional computational cost. Experimental results
on the nuScenes dataset demonstrate the framework's high compatibility with
diverse online perception models. Specifically, it shows remarkable
improvements in HD-map construction and occupancy prediction tasks,
highlighting its potential as a new perception framework for autonomous driving
systems. Our code will be released at
https://github.com/yuantianyuan01/PreSight.