MoReVQA: Exploring Modular Reasoning Models for Video Question Answering
Juhong Min Shyamal Buch Arsha Nagrani Minsu Cho Cordelia Schmid
This paper addresses the task of video question answering (videoQA) via a
decomposed multi-stage, modular reasoning framework. Previous modular methods
have shown promise with a single planning stage ungrounded in visual content.
However, through a simple and effective baseline, we find that such systems can
lead to brittle behavior in practice for challenging videoQA settings. Thus,
unlike traditional single-stage planning methods, we propose a multi-stage
system consisting of an event parser, a grounding stage, and a final reasoning
stage in conjunction with an external memory. All stages are training-free, and
performed using few-shot prompting of large models, creating interpretable
intermediate outputs at each stage. By decomposing the underlying planning and
task complexity, our method, MoReVQA, improves over prior work on standard
videoQA benchmarks (NExT-QA, iVQA, EgoSchema, ActivityNet-QA) with
state-of-the-art results, and extensions to related tasks (grounded videoQA,
paragraph captioning).