MambaPupil: Bidirectional Selective Recurrent model for Event-based Eye
  tracking
Zhong Wang Zengyu Wan Han Han Bohao Liao Yuliang Wu Wei Zhai Yang Cao Zheng-jun Zha
Event-based eye tracking has shown great promise with the high temporal
resolution and low redundancy provided by the event camera. However, the
diversity and abruptness of eye movement patterns, including blinking,
fixating, saccades, and smooth pursuit, pose significant challenges for eye
localization. To achieve a stable event-based eye-tracking system, this paper
proposes a bidirectional long-term sequence modeling and time-varying state
selection mechanism to fully utilize contextual temporal information in
response to the variability of eye movements. Specifically, the MambaPupil
network is proposed, which consists of the multi-layer convolutional encoder to
extract features from the event representations, a bidirectional Gated
Recurrent Unit (GRU), and a Linear Time-Varying State Space Module (LTV-SSM),
to selectively capture contextual correlation from the forward and backward
temporal relationship. Furthermore, the Bina-rep is utilized as a compact event
representation, and the tailor-made data augmentation, called as Event-Cutout,
is proposed to enhance the model's robustness by applying spatial random
masking to the event image. The evaluation on the ThreeET-plus benchmark shows
the superior performance of the MambaPupil, which secured the 1st place in
CVPR'2024 AIS Event-based Eye Tracking challenge.