Video Recognition in Portrait Mode
Mingfei Han Linjie Yang Xiaojie Jin Jiashi Feng Xiaojun Chang Heng Wang
The creation of new datasets often presents new challenges for video
recognition and can inspire novel ideas while addressing these challenges.
While existing datasets mainly comprise landscape mode videos, our paper seeks
to introduce portrait mode videos to the research community and highlight the
unique challenges associated with this video format. With the growing
popularity of smartphones and social media applications, recognizing portrait
mode videos is becoming increasingly important. To this end, we have developed
the first dataset dedicated to portrait mode video recognition, namely
PortraitMode-400. The taxonomy of PortraitMode-400 was constructed in a
data-driven manner, comprising 400 fine-grained categories, and rigorous
quality assurance was implemented to ensure the accuracy of human annotations.
In addition to the new dataset, we conducted a comprehensive analysis of the
impact of video format (portrait mode versus landscape mode) on recognition
accuracy and spatial bias due to the different formats. Furthermore, we
designed extensive experiments to explore key aspects of portrait mode video
recognition, including the choice of data augmentation, evaluation procedure,
the importance of temporal information, and the role of audio modality.
Building on the insights from our experimental results and the introduction of
PortraitMode-400, our paper aims to inspire further research efforts in this
emerging research area.