Semantic Is Enough: Only Semantic Information For NeRF Reconstruction
Ruibo Wang Song Zhang Ping Huang Donghai Zhang Wei Yan
Recent research that combines implicit 3D representation with semantic
information, like Semantic-NeRF, has proven that NeRF model could perform
excellently in rendering 3D structures with semantic labels. This research aims
to extend the Semantic Neural Radiance Fields (Semantic-NeRF) model by focusing
solely on semantic output and removing the RGB output component. We reformulate
the model and its training procedure to leverage only the cross-entropy loss
between the model semantic output and the ground truth semantic images,
removing the colour data traditionally used in the original Semantic-NeRF
approach. We then conduct a series of identical experiments using the original
and the modified Semantic-NeRF model. Our primary objective is to obverse the
impact of this modification on the model performance by Semantic-NeRF, focusing
on tasks such as scene understanding, object detection, and segmentation. The
results offer valuable insights into the new way of rendering the scenes and
provide an avenue for further research and development in semantic-focused 3D
scene understanding.