A Hybrid SNN-ANN Network for Event-based Object Detection with Spatial
  and Temporal Attention
Soikat Hasan Ahmed Jan Finkbeiner Emre Neftci
Event cameras offer high temporal resolution and dynamic range with minimal
motion blur, making them promising for object detection tasks. While Spiking
Neural Networks (SNNs) are a natural match for event-based sensory data and
enable ultra-energy efficient and low latency inference on neuromorphic
hardware, Artificial Neural Networks (ANNs) tend to display more stable
training dynamics and faster convergence resulting in greater task performance.
Hybrid SNN-ANN approaches are a promising alternative, enabling to leverage the
strengths of both SNN and ANN architectures. In this work, we introduce the
first Hybrid Attention-based SNN-ANN backbone for object detection using event
cameras. We propose a novel Attention-based SNN-ANN bridge module to capture
sparse spatial and temporal relations from the SNN layer and convert them into
dense feature maps for the ANN part of the backbone. Experimental results
demonstrate that our proposed method surpasses baseline hybrid and SNN-based
approaches by significant margins, with results comparable to existing
ANN-based methods. Extensive ablation studies confirm the effectiveness of our
proposed modules and architectural choices. These results pave the way toward a
hybrid SNN-ANN architecture that achieves ANN like performance at a drastically
reduced parameter budget. We implemented the SNN blocks on digital neuromorphic
hardware to investigate latency and power consumption and demonstrate the
feasibility of our approach.