Localized Questions in Medical Visual Question Answering
Sergio Tascon-Morales Pablo Márquez-Neila Raphael Sznitman
Visual Question Answering (VQA) models aim to answer natural language
questions about given images. Due to its ability to ask questions that differ
from those used when training the model, medical VQA has received substantial
attention in recent years. However, existing medical VQA models typically focus
on answering questions that refer to an entire image rather than where the
relevant content may be located in the image. Consequently, VQA models are
limited in their interpretability power and the possibility to probe the model
about specific image regions. This paper proposes a novel approach for medical
VQA that addresses this limitation by developing a model that can answer
questions about image regions while considering the context necessary to answer
the questions. Our experimental results demonstrate the effectiveness of our
proposed model, outperforming existing methods on three datasets. Our code and
data are available at https://github.com/sergiotasconmorales/locvqa.