DatasetNeRF: Efficient 3D-aware Data Factory with Generative Radiance
  Fields
Yu Chi Fangneng Zhan Sibo Wu Christian Theobalt Adam Kortylewski
Progress in 3D computer vision tasks demands a huge amount of data, yet
annotating multi-view images with 3D-consistent annotations, or point clouds
with part segmentation is both time-consuming and challenging. This paper
introduces DatasetNeRF, a novel approach capable of generating infinite,
high-quality 3D-consistent 2D annotations alongside 3D point cloud
segmentations, while utilizing minimal 2D human-labeled annotations.
Specifically, we leverage the strong semantic prior within a 3D generative
model to train a semantic decoder, requiring only a handful of fine-grained
labeled samples. Once trained, the decoder efficiently generalizes across the
latent space, enabling the generation of infinite data. The generated data is
applicable across various computer vision tasks, including video segmentation
and 3D point cloud segmentation. Our approach not only surpasses baseline
models in segmentation quality, achieving superior 3D consistency and
segmentation precision on individual images, but also demonstrates versatility
by being applicable to both articulated and non-articulated generative models.
Furthermore, we explore applications stemming from our approach, such as
3D-aware semantic editing and 3D inversion.