REACT: Recognize Every Action Everywhere All At Once
Naga VS Raviteja Chappa Pha Nguyen Page Daniel Dobbs Khoa Luu
Group Activity Recognition (GAR) is a fundamental problem in computer vision,
with diverse applications in sports video analysis, video surveillance, and
social scene understanding. Unlike conventional action recognition, GAR aims to
classify the actions of a group of individuals as a whole, requiring a deep
understanding of their interactions and spatiotemporal relationships. To
address the challenges in GAR, we present REACT (\textbf{R}ecognize
\textbf{E}very \textbf{Act}ion Everywhere All At Once), a novel architecture
inspired by the transformer encoder-decoder model explicitly designed to model
complex contextual relationships within videos, including multi-modality and
spatio-temporal features. Our architecture features a cutting-edge
Vision-Language Encoder block for integrated temporal, spatial, and multi-modal
interaction modeling. This component efficiently encodes spatiotemporal
interactions, even with sparsely sampled frames, and recovers essential local
information. Our Action Decoder Block refines the joint understanding of text
and video data, allowing us to precisely retrieve bounding boxes, enhancing the
link between semantics and visual reality. At the core, our Actor Fusion Block
orchestrates a fusion of actor-specific data and textual features, striking a
balance between specificity and context. Our method outperforms
state-of-the-art GAR approaches in extensive experiments, demonstrating
superior accuracy in recognizing and understanding group activities. Our
architecture's potential extends to diverse real-world applications, offering
empirical evidence of its performance gains. This work significantly advances
the field of group activity recognition, providing a robust framework for
nuanced scene comprehension.