Cross-Modality Gait Recognition: Bridging LiDAR and Camera Modalities
  for Human Identification
Rui Wang Chuanfu Shen Manuel J. Marin-Jimenez George Q. Huang Shiqi Yu
Current gait recognition research mainly focuses on identifying pedestrians
captured by the same type of sensor, neglecting the fact that individuals may
be captured by different sensors in order to adapt to various environments. A
more practical approach should involve cross-modality matching across different
sensors. Hence, this paper focuses on investigating the problem of
cross-modality gait recognition, with the objective of accurately identifying
pedestrians across diverse vision sensors. We present CrossGait inspired by the
feature alignment strategy, capable of cross retrieving diverse data
modalities. Specifically, we investigate the cross-modality recognition task by
initially extracting features within each modality and subsequently aligning
these features across modalities. To further enhance the cross-modality
performance, we propose a Prototypical Modality-shared Attention Module that
learns modality-shared features from two modality-specific features.
Additionally, we design a Cross-modality Feature Adapter that transforms the
learned modality-specific features into a unified feature space. Extensive
experiments conducted on the SUSTech1K dataset demonstrate the effectiveness of
CrossGait: (1) it exhibits promising cross-modality ability in retrieving
pedestrians across various modalities from different sensors in diverse scenes,
and (2) CrossGait not only learns modality-shared features for cross-modality
gait recognition but also maintains modality-specific features for
single-modality recognition.