Sub-action Prototype Learning for Point-level Weakly-supervised Temporal
  Action Localization
Yueyang Li Yonghong Hou Wanqing Li
Point-level weakly-supervised temporal action localization (PWTAL) aims to
localize actions with only a single timestamp annotation for each action
instance. Existing methods tend to mine dense pseudo labels to alleviate the
label sparsity, but overlook the potential sub-action temporal structures,
resulting in inferior performance. To tackle this problem, we propose a novel
sub-action prototype learning framework (SPL-Loc) which comprises Sub-action
Prototype Clustering (SPC) and Ordered Prototype Alignment (OPA). SPC
adaptively extracts representative sub-action prototypes which are capable to
perceive the temporal scale and spatial content variation of action instances.
OPA selects relevant prototypes to provide completeness clue for pseudo label
generation by applying a temporal alignment loss. As a result, pseudo labels
are derived from alignment results to improve action boundary prediction.
Extensive experiments on three popular benchmarks demonstrate that the proposed
SPL-Loc significantly outperforms existing SOTA PWTAL methods.