MoE-TinyMed: Mixture of Experts for Tiny Medical Large Vision-Language
  Models
Songtao Jiang Tuo Zheng Yan Zhang Yeying Jin Zuozhu Liu
Mixture of Expert Tuning (MoE-Tuning) has effectively enhanced the
performance of general MLLMs with fewer parameters, yet its application in
resource-limited medical settings has not been fully explored. To address this
gap, we developed MoE-TinyMed, a model tailored for medical applications that
significantly lowers parameter demands. In evaluations on the VQA-RAD, SLAKE,
and Path-VQA datasets, MoE-TinyMed outperformed LLaVA-Med in all Med-VQA closed
settings with just 3.6B parameters. Additionally, a streamlined version with 2B
parameters surpassed LLaVA-Med's performance in PathVQA, showcasing its
effectiveness in resource-limited healthcare settings.