SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video
  Grounding
Wenrui Li Xiaopeng Hong Xiaopeng Fan
Temporal video grounding (TVG) is a critical task in video content
understanding. Despite significant advancements, existing methods often limit
in capturing the fine-grained relationships between multimodal inputs and the
high computational costs with processing long video sequences. To address these
limitations, we introduce a novel SpikeMba: multi-modal spiking saliency mamba
for temporal video grounding. In our work, we integrate the Spiking Neural
Networks (SNNs) and state space models (SSMs) to capture the fine-grained
relationships of multimodal features effectively. Specifically, we introduce
the relevant slots to enhance the model's memory capabilities, enabling a
deeper contextual understanding of video sequences. The contextual moment
reasoner leverages these slots to maintain a balance between contextual
information preservation and semantic relevance exploration. Simultaneously,
the spiking saliency detector capitalizes on the unique properties of SNNs to
accurately locate salient proposals. Our experiments demonstrate the
effectiveness of SpikeMba, which consistently outperforms state-of-the-art
methods across mainstream benchmarks.