Variable Radiance Field for Real-Life Category-Specifc Reconstruction
  from Single Image
Kun Wang Zhiqiang Yan Zhenyu Zhang Xiang Li Jun Li Jian Yang
Reconstructing category-specific objects from a single image is a challenging
task that requires inferring the geometry and appearance of an object from a
limited viewpoint. Existing methods typically rely on local feature retrieval
based on re-projection with known camera intrinsic, which are slow and prone to
distortion at viewpoints distant from the input image. In this paper, we
present Variable Radiance Field (VRF), a novel framework that can efficiently
reconstruct category-specific objects from a single image without known camera
parameters. Our key contributions are: (1) We parameterize the geometry and
appearance of the object using a multi-scale global feature extractor, which
avoids frequent point-wise feature retrieval and camera dependency. We also
propose a contrastive learning-based pretraining strategy to improve the
feature extractor. (2) We reduce the geometric complexity of the object by
learning a category template, and use hypernetworks to generate a small neural
radiance field for fast and instance-specific rendering. (3) We align each
training instance to the template space using a learned similarity
transformation, which enables semantic-consistent learning across different
objects. We evaluate our method on the CO3D dataset and show that it
outperforms existing methods in terms of quality and speed. We also demonstrate
its applicability to shape interpolation and object placement tasks.