Efficient Deformable Tissue Reconstruction via Orthogonal Neural Plane
Chen Yang Kailing Wang Yuehao Wang Qi Dou Xiaokang Yang Wei Shen
Intraoperative imaging techniques for reconstructing deformable tissues in
vivo are pivotal for advanced surgical systems. Existing methods either
compromise on rendering quality or are excessively computationally intensive,
often demanding dozens of hours to perform, which significantly hinders their
practical application. In this paper, we introduce Fast Orthogonal Plane
(Forplane), a novel, efficient framework based on neural radiance fields (NeRF)
for the reconstruction of deformable tissues. We conceptualize surgical
procedures as 4D volumes, and break them down into static and dynamic fields
comprised of orthogonal neural planes. This factorization iscretizes the
four-dimensional space, leading to a decreased memory usage and faster
optimization. A spatiotemporal importance sampling scheme is introduced to
improve performance in regions with tool occlusion as well as large motions and
accelerate training. An efficient ray marching method is applied to skip
sampling among empty regions, significantly improving inference speed. Forplane
accommodates both binocular and monocular endoscopy videos, demonstrating its
extensive applicability and flexibility. Our experiments, carried out on two in
vivo datasets, the EndoNeRF and Hamlyn datasets, demonstrate the effectiveness
of our framework. In all cases, Forplane substantially accelerates both the
optimization process (by over 100 times) and the inference process (by over 15
times) while maintaining or even improving the quality across a variety of
non-rigid deformations. This significant performance improvement promises to be
a valuable asset for future intraoperative surgical applications. The code of
our project is now available at https://github.com/Loping151/ForPlane.