CharNeRF: 3D Character Generation from Concept Art
Eddy Chu Yiyang Chen Chedy Raissi Anand Bhojan
3D modeling holds significant importance in the realms of AR/VR and gaming,
allowing for both artistic creativity and practical applications. However, the
process is often time-consuming and demands a high level of skill. In this
paper, we present a novel approach to create volumetric representations of 3D
characters from consistent turnaround concept art, which serves as the standard
input in the 3D modeling industry. While Neural Radiance Field (NeRF) has been
a game-changer in image-based 3D reconstruction, to the best of our knowledge,
there is no known research that optimizes the pipeline for concept art. To
harness the potential of concept art, with its defined body poses and specific
view angles, we propose encoding it as priors for our model. We train the
network to make use of these priors for various 3D points through a learnable
view-direction-attended multi-head self-attention layer. Additionally, we
demonstrate that a combination of ray sampling and surface sampling enhances
the inference capabilities of our network. Our model is able to generate
high-quality 360-degree views of characters. Subsequently, we provide a simple
guideline to better leverage our model to extract the 3D mesh. It is important
to note that our model's inferencing capabilities are influenced by the
training data's characteristics, primarily focusing on characters with a single
head, two arms, and two legs. Nevertheless, our methodology remains versatile
and adaptable to concept art from diverse subject matters, without imposing any
specific assumptions on the data.