High-fidelity 3D Reconstruction of Plants using Neural Radiance Field
Kewei Hu Ying Wei Yaoqiang Pan Hanwen Kang Chao Chen
Accurate reconstruction of plant phenotypes plays a key role in optimising
sustainable farming practices in the field of Precision Agriculture (PA).
Currently, optical sensor-based approaches dominate the field, but the need for
high-fidelity 3D reconstruction of crops and plants in unstructured
agricultural environments remains challenging. Recently, a promising
development has emerged in the form of Neural Radiance Field (NeRF), a novel
method that utilises neural density fields. This technique has shown impressive
performance in various novel vision synthesis tasks, but has remained
relatively unexplored in the agricultural context. In our study, we focus on
two fundamental tasks within plant phenotyping: (1) the synthesis of 2D
novel-view images and (2) the 3D reconstruction of crop and plant models. We
explore the world of neural radiance fields, in particular two SOTA methods:
Instant-NGP, which excels in generating high-quality images with impressive
training and inference speed, and Instant-NSR, which improves the reconstructed
geometry by incorporating the Signed Distance Function (SDF) during training.
In particular, we present a novel plant phenotype dataset comprising real plant
images from production environments. This dataset is a first-of-its-kind
initiative aimed at comprehensively exploring the advantages and limitations of
NeRF in agricultural contexts. Our experimental results show that NeRF
demonstrates commendable performance in the synthesis of novel-view images and
is able to achieve reconstruction results that are competitive with Reality
Capture, a leading commercial software for 3D Multi-View Stereo (MVS)-based
reconstruction. However, our study also highlights certain drawbacks of NeRF,
including relatively slow training speeds, performance limitations in cases of
insufficient sampling, and challenges in obtaining geometry quality in complex
setups.