STAIR: Semantic-Targeted Active Implicit Reconstruction
Liren Jin Haofei Kuang Yue Pan Cyrill Stachniss Marija Popović
Many autonomous robotic applications require object-level understanding when
deployed. Actively reconstructing objects of interest, i.e. objects with
specific semantic meanings, is therefore relevant for a robot to perform
downstream tasks in an initially unknown environment. In this work, we propose
a novel framework for semantic-targeted active reconstruction using posed RGB-D
measurements and 2D semantic labels as input. The key components of our
framework are a semantic implicit neural representation and a compatible
planning utility function based on semantic rendering and uncertainty
estimation, enabling adaptive view planning to target objects of interest. Our
planning approach achieves better reconstruction performance in terms of mesh
and novel view rendering quality compared to implicit reconstruction baselines
that do not consider semantics for view planning. Our framework further
outperforms a state-of-the-art semantic-targeted active reconstruction pipeline
based on explicit maps, justifying our choice of utilising implicit neural
representations to tackle semantic-targeted active reconstruction problems.