GigaPevt: Multimodal Medical Assistant
Pavel Blinov Konstantin Egorov Ivan Sviridov Nikolay Ivanov Stepan Botman Evgeniy Tagin Stepan Kudin Galina Zubkova Andrey Savchenko
Building an intelligent and efficient medical assistant is still a
challenging AI problem. The major limitation comes from the data modality
scarceness, which reduces comprehensive patient perception. This demo paper
presents the GigaPevt, the first multimodal medical assistant that combines the
dialog capabilities of large language models with specialized medical models.
Such an approach shows immediate advantages in dialog quality and metric
performance, with a 1.18\% accuracy improvement in the question-answering task.