SlowFast Network for Continuous Sign Language Recognition
Junseok Ahn Youngjoon Jang Joon Son Chung
The objective of this work is the effective extraction of spatial and dynamic
features for Continuous Sign Language Recognition (CSLR). To accomplish this,
we utilise a two-pathway SlowFast network, where each pathway operates at
distinct temporal resolutions to separately capture spatial (hand shapes,
facial expressions) and dynamic (movements) information. In addition, we
introduce two distinct feature fusion methods, carefully designed for the
characteristics of CSLR: (1) Bi-directional Feature Fusion (BFF), which
facilitates the transfer of dynamic semantics into spatial semantics and vice
versa; and (2) Pathway Feature Enhancement (PFE), which enriches dynamic and
spatial representations through auxiliary subnetworks, while avoiding the need
for extra inference time. As a result, our model further strengthens spatial
and dynamic representations in parallel. We demonstrate that the proposed
framework outperforms the current state-of-the-art performance on popular CSLR
datasets, including PHOENIX14, PHOENIX14-T, and CSL-Daily.