Iterative Superquadric Recomposition of 3D Objects from Multiple Views
Stephan Alaniz Massimiliano Mancini Zeynep Akata
Humans are good at recomposing novel objects, i.e. they can identify
commonalities between unknown objects from general structure to finer detail,
an ability difficult to replicate by machines. We propose a framework, ISCO, to
recompose an object using 3D superquadrics as semantic parts directly from 2D
views without training a model that uses 3D supervision. To achieve this, we
optimize the superquadric parameters that compose a specific instance of the
object, comparing its rendered 3D view and 2D image silhouette. Our ISCO
framework iteratively adds new superquadrics wherever the reconstruction error
is high, abstracting first coarse regions and then finer details of the target
object. With this simple coarse-to-fine inductive bias, ISCO provides
consistent superquadrics for related object parts, despite not having any
semantic supervision. Since ISCO does not train any neural network, it is also
inherently robust to out-of-distribution objects. Experiments show that,
compared to recent single instance superquadrics reconstruction approaches,
ISCO provides consistently more accurate 3D reconstructions, even from images
in the wild. Code available at https://github.com/ExplainableML/ISCO .