SRFNet: Monocular Depth Estimation with Fine-grained Structure via
  Spatial Reliability-oriented Fusion of Frames and Events
Tianbo Pan Zidong Cao Lin Wang
Monocular depth estimation is a crucial task to measure distance relative to
a camera, which is important for applications, such as robot navigation and
self-driving. Traditional frame-based methods suffer from performance drops due
to the limited dynamic range and motion blur. Therefore, recent works leverage
novel event cameras to complement or guide the frame modality via frame-event
feature fusion. However, event streams exhibit spatial sparsity, leaving some
areas unperceived, especially in regions with marginal light changes.
Therefore, direct fusion methods, e.g., RAMNet, often ignore the contribution
of the most confident regions of each modality. This leads to structural
ambiguity in the modality fusion process, thus degrading the depth estimation
performance. In this paper, we propose a novel Spatial Reliability-oriented
Fusion Network (SRFNet), that can estimate depth with fine-grained structure at
both daytime and nighttime. Our method consists of two key technical
components. Firstly, we propose an attention-based interactive fusion (AIF)
module that applies spatial priors of events and frames as the initial masks
and learns the consensus regions to guide the inter-modal feature fusion. The
fused feature are then fed back to enhance the frame and event feature
learning. Meanwhile, it utilizes an output head to generate a fused mask, which
is iteratively updated for learning consensual spatial priors. Secondly, we
propose the Reliability-oriented Depth Refinement (RDR) module to estimate
dense depth with the fine-grained structure based on the fused features and
masks. We evaluate the effectiveness of our method on the synthetic and
real-world datasets, which shows that, even without pretraining, our method
outperforms the prior methods, e.g., RAMNet, especially in night scenes. Our
project homepage: https://vlislab22.github.io/SRFNet.