Answering from Sure to Uncertain: Uncertainty-Aware Curriculum Learning
  for Video Question Answering
Haopeng Li Qiuhong Ke Mingming Gong Tom Drummond
While significant advancements have been made in video question answering
(VideoQA), the potential benefits of enhancing model generalization through
tailored difficulty scheduling have been largely overlooked in existing
research. This paper seeks to bridge that gap by incorporating VideoQA into a
curriculum learning (CL) framework that progressively trains models from
simpler to more complex data. Recognizing that conventional self-paced CL
methods rely on training loss for difficulty measurement, which might not
accurately reflect the intricacies of video-question pairs, we introduce the
concept of uncertainty-aware CL. Here, uncertainty serves as the guiding
principle for dynamically adjusting the difficulty. Furthermore, we address the
challenge posed by uncertainty by presenting a probabilistic modeling approach
for VideoQA. Specifically, we conceptualize VideoQA as a stochastic computation
graph, where the hidden representations are treated as stochastic variables.
This yields two distinct types of uncertainty: one related to the inherent
uncertainty in the data and another pertaining to the model's confidence. In
practice, we seamlessly integrate the VideoQA model into our framework and
conduct comprehensive experiments. The findings affirm that our approach not
only achieves enhanced performance but also effectively quantifies uncertainty
in the context of VideoQA.