STF: Spatio-Temporal Fusion Module for Improving Video Object Detection
Noreen Anwar Guillaume-Alexandre Bilodeau Wassim Bouachir
Consecutive frames in a video contain redundancy, but they may also contain
relevant complementary information for the detection task. The objective of our
work is to leverage this complementary information to improve detection.
Therefore, we propose a spatio-temporal fusion framework (STF). We first
introduce multi-frame and single-frame attention modules that allow a neural
network to share feature maps between nearby frames to obtain more robust
object representations. Second, we introduce a dual-frame fusion module that
merges feature maps in a learnable manner to improve them. Our evaluation is
conducted on three different benchmarks including video sequences of moving
road users. The performed experiments demonstrate that the proposed
spatio-temporal fusion module leads to improved detection performance compared
to baseline object detectors. Code is available at
https://github.com/noreenanwar/STF-module