Exploiting Priors from 3D Diffusion Models for RGB-Based One-Shot View
  Planning
Sicong Pan Liren Jin Xuying Huang Cyrill Stachniss Marija Popović Maren Bennewitz
Object reconstruction is relevant for many autonomous robotic tasks that
require interaction with the environment. A key challenge in such scenarios is
planning view configurations to collect informative measurements for
reconstructing an initially unknown object. One-shot view planning enables
efficient data collection by predicting view configurations and planning the
globally shortest path connecting all views at once. However, geometric priors
about the object are required to conduct one-shot view planning. In this work,
we propose a novel one-shot view planning approach that utilizes the powerful
3D generation capabilities of diffusion models as priors. By incorporating such
geometric priors into our pipeline, we achieve effective one-shot view planning
starting with only a single RGB image of the object to be reconstructed. Our
planning experiments in simulation and real-world setups indicate that our
approach balances well between object reconstruction quality and movement cost.