Learning Effective NeRFs and SDFs Representations with 3D Generative
  Adversarial Networks for 3D Object Generation: Technical Report for ICCV 2023
  OmniObject3D Challenge
Zheyuan Yang Yibo Liu Guile Wu Tongtong Cao Yuan Ren Yang Liu Bingbing Liu
In this technical report, we present a solution for 3D object generation of
ICCV 2023 OmniObject3D Challenge. In recent years, 3D object generation has
made great process and achieved promising results, but it remains a challenging
task due to the difficulty of generating complex, textured and high-fidelity
results. To resolve this problem, we study learning effective NeRFs and SDFs
representations with 3D Generative Adversarial Networks (GANs) for 3D object
generation. Specifically, inspired by recent works, we use the efficient
geometry-aware 3D GANs as the backbone incorporating with label embedding and
color mapping, which enables to train the model on different taxonomies
simultaneously. Then, through a decoder, we aggregate the resulting features to
generate Neural Radiance Fields (NeRFs) based representations for rendering
high-fidelity synthetic images. Meanwhile, we optimize Signed Distance
Functions (SDFs) to effectively represent objects with 3D meshes. Besides, we
observe that this model can be effectively trained with only a few images of
each object from a variety of classes, instead of using a great number of
images per object or training one model per class. With this pipeline, we can
optimize an effective model for 3D object generation. This solution is one of
the final top-3-place solutions in the ICCV 2023 OmniObject3D Challenge.