Spike-NeRF: Neural Radiance Field Based On Spike Camera
Yijia Guo Yuanxi Bai Liwen Hu Mianzhi Liu Ziyi Guo Lei Ma Tiejun Huang
As a neuromorphic sensor with high temporal resolution, spike cameras offer
notable advantages over traditional cameras in high-speed vision applications
such as high-speed optical estimation, depth estimation, and object tracking.
Inspired by the success of the spike camera, we proposed Spike-NeRF, the first
Neural Radiance Field derived from spike data, to achieve 3D reconstruction and
novel viewpoint synthesis of high-speed scenes. Instead of the multi-view
images at the same time of NeRF, the inputs of Spike-NeRF are continuous spike
streams captured by a moving spike camera in a very short time. To reconstruct
a correct and stable 3D scene from high-frequency but unstable spike data, we
devised spike masks along with a distinctive loss function. We evaluate our
method qualitatively and numerically on several challenging synthetic scenes
generated by blender with the spike camera simulator. Our results demonstrate
that Spike-NeRF produces more visually appealing results than the existing
methods and the baseline we proposed in high-speed scenes. Our code and data
will be released soon.