GET: Group Event Transformer for Event-Based Vision
Yansong Peng Yueyi Zhang Zhiwei Xiong Xiaoyan Sun Feng Wu
Event cameras are a type of novel neuromorphic sen-sor that has been gaining
increasing attention. Existing event-based backbones mainly rely on image-based
designs to extract spatial information within the image transformed from
events, overlooking important event properties like time and polarity. To
address this issue, we propose a novel Group-based vision Transformer backbone
for Event-based vision, called Group Event Transformer (GET), which de-couples
temporal-polarity information from spatial infor-mation throughout the feature
extraction process. Specifi-cally, we first propose a new event representation
for GET, named Group Token, which groups asynchronous events based on their
timestamps and polarities. Then, GET ap-plies the Event Dual Self-Attention
block, and Group Token Aggregation module to facilitate effective feature
commu-nication and integration in both the spatial and temporal-polarity
domains. After that, GET can be integrated with different downstream tasks by
connecting it with vari-ous heads. We evaluate our method on four event-based
classification datasets (Cifar10-DVS, N-MNIST, N-CARS, and DVS128Gesture) and
two event-based object detection datasets (1Mpx and Gen1), and the results
demonstrate that GET outperforms other state-of-the-art methods. The code is
available at https://github.com/Peterande/GET-Group-Event-Transformer.