Tracking-Assisted Object Detection with Event Cameras
Ting-Kang Yen Igor Morawski Shusil Dangi Kai He Chung-Yi Lin Jia-Fong Yeh Hung-Ting Su Winston Hsu
Event-based object detection has recently garnered attention in the computer
vision community due to the exceptional properties of event cameras, such as
high dynamic range and no motion blur. However, feature asynchronism and
sparsity cause invisible objects due to no relative motion to the camera,
posing a significant challenge in the task. Prior works have studied various
memory mechanisms to preserve as many features as possible at the current time,
guided by temporal clues. While these implicit-learned memories retain some
short-term information, they still struggle to preserve long-term features
effectively. In this paper, we consider those invisible objects as
pseudo-occluded objects and aim to reveal their features. Firstly, we introduce
visibility attribute of objects and contribute an auto-labeling algorithm to
append additional visibility labels on an existing event camera dataset.
Secondly, we exploit tracking strategies for pseudo-occluded objects to
maintain their permanence and retain their bounding boxes, even when features
have not been available for a very long time. These strategies can be treated
as an explicit-learned memory guided by the tracking objective to record the
displacements of objects across frames. Lastly, we propose a spatio-temporal
feature aggregation module to enrich the latent features and a consistency loss
to increase the robustness of the overall pipeline. We conduct comprehensive
experiments to verify our method's effectiveness where still objects are
retained but real occluded objects are discarded. The results demonstrate that
(1) the additional visibility labels can assist in supervised training, and (2)
our method outperforms state-of-the-art approaches with a significant
improvement of 7.9% absolute mAP.