Object-Centric Domain Randomization for 3D Shape Reconstruction in the
  Wild
Junhyeong Cho Kim Youwang Hunmin Yang Tae-Hyun Oh
One of the biggest challenges in single-view 3D shape reconstruction in the
wild is the scarcity of <3D shape, 2D image>-paired data from real-world
environments. Inspired by remarkable achievements via domain randomization, we
propose ObjectDR which synthesizes such paired data via a random simulation of
visual variations in object appearances and backgrounds. Our data synthesis
framework exploits a conditional generative model (e.g., ControlNet) to
generate images conforming to spatial conditions such as 2.5D sketches, which
are obtainable through a rendering process of 3D shapes from object collections
(e.g., Objaverse-XL). To simulate diverse variations while preserving object
silhouettes embedded in spatial conditions, we also introduce a disentangled
framework which leverages an initial object guidance. After synthesizing a wide
range of data, we pre-train a model on them so that it learns to capture a
domain-invariant geometry prior which is consistent across various domains. We
validate its effectiveness by substantially improving 3D shape reconstruction
models on a real-world benchmark. In a scale-up evaluation, our pre-training
achieves 23.6% superior results compared with the pre-training on high-quality
computer graphics renderings.