Query-aware Long Video Localization and Relation Discrimination for Deep
  Video Understanding
Yuanxing Xu Yuting Wei Bin Wu
The surge in video and social media content underscores the need for a deeper
understanding of multimedia data. Most of the existing mature video
understanding techniques perform well with short formats and content that
requires only shallow understanding, but do not perform well with long format
videos that require deep understanding and reasoning. Deep Video Understanding
(DVU) Challenge aims to push the boundaries of multimodal extraction, fusion,
and analytics to address the problem of holistically analyzing long videos and
extract useful knowledge to solve different types of queries. This paper
introduces a query-aware method for long video localization and relation
discrimination, leveraging an imagelanguage pretrained model. This model
adeptly selects frames pertinent to queries, obviating the need for a complete
movie-level knowledge graph. Our approach achieved first and fourth positions
for two groups of movie-level queries. Sufficient experiments and final
rankings demonstrate its effectiveness and robustness.