3DMiner: Discovering Shapes from Large-Scale Unannotated Image Datasets
Ta-Ying Cheng Matheus Gadelha Soren Pirk Thibault Groueix Radomir Mech Andrew Markham Niki Trigoni
We present 3DMiner -- a pipeline for mining 3D shapes from challenging
large-scale unannotated image datasets. Unlike other unsupervised 3D
reconstruction methods, we assume that, within a large-enough dataset, there
must exist images of objects with similar shapes but varying backgrounds,
textures, and viewpoints. Our approach leverages the recent advances in
learning self-supervised image representations to cluster images with
geometrically similar shapes and find common image correspondences between
them. We then exploit these correspondences to obtain rough camera estimates as
initialization for bundle-adjustment. Finally, for every image cluster, we
apply a progressive bundle-adjusting reconstruction method to learn a neural
occupancy field representing the underlying shape. We show that this procedure
is robust to several types of errors introduced in previous steps (e.g., wrong
camera poses, images containing dissimilar shapes, etc.), allowing us to obtain
shape and pose annotations for images in-the-wild. When using images from Pix3D
chairs, our method is capable of producing significantly better results than
state-of-the-art unsupervised 3D reconstruction techniques, both quantitatively
and qualitatively. Furthermore, we show how 3DMiner can be applied to
in-the-wild data by reconstructing shapes present in images from the LAION-5B
dataset. Project Page: https://ttchengab.github.io/3dminerOfficial