Spatio-Temporal Proximity-Aware Dual-Path Model for Panoramic Activity
  Recognition
Sumin Lee Yooseung Wang Sangmin Woo Changick Kim
Panoramic Activity Recognition (PAR) seeks to identify diverse human
activities across different scales, from individual actions to social group and
global activities in crowded panoramic scenes. PAR presents two major
challenges: 1) recognizing the nuanced interactions among numerous individuals
and 2) understanding multi-granular human activities. To address these, we
propose Social Proximity-aware Dual-Path Network (SPDP-Net) based on two key
design principles. First, while previous works often focus on spatial distance
among individuals within an image, we argue to consider the spatio-temporal
proximity. It is crucial for individual relation encoding to correctly
understand social dynamics. Secondly, deviating from existing hierarchical
approaches (individual-to-social-to-global activity), we introduce a dual-path
architecture for multi-granular activity recognition. This architecture
comprises individual-to-global and individual-to-social paths, mutually
reinforcing each other's task with global-local context through multiple
layers. Through extensive experiments, we validate the effectiveness of the
spatio-temporal proximity among individuals and the dual-path architecture in
PAR. Furthermore, SPDP-Net achieves new state-of-the-art performance with
46.5\% of overall F1 score on JRDB-PAR dataset.