Detecting Every Object from Events
Haitian Zhang Chang Xu Xinya Wang Bingde Liu Guang Hua Lei Yu Wen Yang
Object detection is critical in autonomous driving, and it is more practical
yet challenging to localize objects of unknown categories: an endeavour known
as Class-Agnostic Object Detection (CAOD). Existing studies on CAOD
predominantly rely on ordinary cameras, but these frame-based sensors usually
have high latency and limited dynamic range, leading to safety risks in
real-world scenarios. In this study, we turn to a new modality enabled by the
so-called event camera, featured by its sub-millisecond latency and high
dynamic range, for robust CAOD. We propose Detecting Every Object in Events
(DEOE), an approach tailored for achieving high-speed, class-agnostic
open-world object detection in event-based vision. Built upon the fast
event-based backbone: recurrent vision transformer, we jointly consider the
spatial and temporal consistencies to identify potential objects. The
discovered potential objects are assimilated as soft positive samples to avoid
being suppressed as background. Moreover, we introduce a disentangled
objectness head to separate the foreground-background classification and novel
object discovery tasks, enhancing the model's generalization in localizing
novel objects while maintaining a strong ability to filter out the background.
Extensive experiments confirm the superiority of our proposed DEOE in
comparison with three strong baseline methods that integrate the
state-of-the-art event-based object detector with advancements in RGB-based
CAOD. Our code is available at https://github.com/Hatins/DEOE.