HAtt-Flow: Hierarchical Attention-Flow Mechanism for Group Activity
  Scene Graph Generation in Videos
Naga VS Raviteja Chappa Pha Nguyen Thi Hoang Ngan Le Khoa Luu
Group Activity Scene Graph (GASG) generation is a challenging task in
computer vision, aiming to anticipate and describe relationships between
subjects and objects in video sequences. Traditional Video Scene Graph
Generation (VidSGG) methods focus on retrospective analysis, limiting their
predictive capabilities. To enrich the scene understanding capabilities, we
introduced a GASG dataset extending the JRDB dataset with nuanced annotations
involving \textit{Appearance, Interaction, Position, Relationship, and
Situation} attributes. This work also introduces an innovative approach,
\textbf{H}ierarchical \textbf{Att}ention-\textbf{Flow} (HAtt-Flow) Mechanism,
rooted in flow network theory to enhance GASG performance. Flow-Attention
incorporates flow conservation principles, fostering competition for sources
and allocation for sinks, effectively preventing the generation of trivial
attention. Our proposed approach offers a unique perspective on attention
mechanisms, where conventional "values" and "keys" are transformed into sources
and sinks, respectively, creating a novel framework for attention-based models.
Through extensive experiments, we demonstrate the effectiveness of our
Hatt-Flow model and the superiority of our proposed Flow-Attention mechanism.
This work represents a significant advancement in predictive video scene
understanding, providing valuable insights and techniques for applications that
require real-time relationship prediction in video data.