Visual Question Generation in Bengali
Mahmud Hasan Labiba Islam Jannatul Ferdous Ruma Tasmiah Tahsin Mayeesha Rashedur M. Rahman
The task of Visual Question Generation (VQG) is to generate human-like
questions relevant to the given image. As VQG is an emerging research field,
existing works tend to focus only on resource-rich language such as English due
to the availability of datasets. In this paper, we propose the first Bengali
Visual Question Generation task and develop a novel transformer-based
encoder-decoder architecture that generates questions in Bengali when given an
image. We propose multiple variants of models - (i) image-only: baseline model
of generating questions from images without additional information, (ii)
image-category and image-answer-category: guided VQG where we condition the
model to generate questions based on the answer and the category of expected
question. These models are trained and evaluated on the translated VQAv2.0
dataset. Our quantitative and qualitative results establish the first state of
the art models for VQG task in Bengali and demonstrate that our models are
capable of generating grammatically correct and relevant questions. Our
quantitative results show that our image-cat model achieves a BLUE-1 score of
33.12 and BLEU-3 score of 7.56 which is the highest of the other two variants.
We also perform a human evaluation to assess the quality of the generation
tasks. Human evaluation suggests that image-cat model is capable of generating
goal-driven and attribute-specific questions and also stays relevant to the
corresponding image.