LABELMAKER: Automatic Semantic Label Generation from RGB-D Trajectories
Silvan Weder Hermann Blum Francis Engelmann Marc Pollefeys
Semantic annotations are indispensable to train or evaluate perception
models, yet very costly to acquire. This work introduces a fully automated
2D/3D labeling framework that, without any human intervention, can generate
labels for RGB-D scans at equal (or better) level of accuracy than comparable
manually annotated datasets such as ScanNet. Our approach is based on an
ensemble of state-of-the-art segmentation models and 3D lifting through neural
rendering. We demonstrate the effectiveness of our LabelMaker pipeline by
generating significantly better labels for the ScanNet datasets and
automatically labelling the previously unlabeled ARKitScenes dataset. Code and
models are available at https://labelmaker.org