Generative Visual Question Answering
Ethan Shen Scotty Singh Bhavesh Kumar
Multi-modal tasks involving vision and language in deep learning continue to
rise in popularity and are leading to the development of newer models that can
generalize beyond the extent of their training data. The current models lack
temporal generalization which enables models to adapt to changes in future
data. This paper discusses a viable approach to creating an advanced Visual
Question Answering (VQA) model which can produce successful results on temporal
generalization. We propose a new data set, GenVQA, utilizing images and
captions from the VQAv2 and MS-COCO dataset to generate new images through
stable diffusion. This augmented dataset is then used to test a combination of
seven baseline and cutting edge VQA models. Performance evaluation focuses on
questions mirroring the original VQAv2 dataset, with the answers having been
adjusted to the new images. This paper's purpose is to investigate the
robustness of several successful VQA models to assess their performance on
future data distributions. Model architectures are analyzed to identify common
stylistic choices that improve generalization under temporal distribution
shifts. This research highlights the importance of creating a large-scale
future shifted dataset. This data can enhance the robustness of VQA models,
allowing their future peers to have improved ability to adapt to temporal
distribution shifts.