Free Form Medical Visual Question Answering in Radiology
Abhishek Narayanan Rushabh Musthyala Rahul Sankar Anirudh Prasad Nistala Pranav Singh Jacopo Cirrone
Visual Question Answering (VQA) in the medical domain presents a unique,
interdisciplinary challenge, combining fields such as Computer Vision, Natural
Language Processing, and Knowledge Representation. Despite its importance,
research in medical VQA has been scant, only gaining momentum since 2018.
Addressing this gap, our research delves into the effective representation of
radiology images and the joint learning of multimodal representations,
surpassing existing methods. We innovatively augment the SLAKE dataset,
enabling our model to respond to a more diverse array of questions, not limited
to the immediate content of radiology or pathology images. Our model achieves a
top-1 accuracy of 79.55\% with a less complex architecture, demonstrating
comparable performance to current state-of-the-art models. This research not
only advances medical VQA but also opens avenues for practical applications in
diagnostic settings.