UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic
  Cross-modal Learnable Prompts
Chenlu Zhan Yufei Zhang Yu Lin Gaoang Wang Hongwei Wang
Medical vision-language pre-training (Med-VLP) models have recently
accelerated the fast-growing medical diagnostics application. However, most
Med-VLP models learn task-specific representations independently from scratch,
thereby leading to great inflexibility when they work across multiple
fine-tuning tasks. In this work, we propose UniDCP, a Unified medical
vision-language model with Dynamic Cross-modal learnable Prompts, which can be
plastically applied to multiple medical vision-language tasks. Specifically, we
explicitly construct a unified framework to harmonize diverse inputs from
multiple pretraining tasks by leveraging cross-modal prompts for unification,
which accordingly can accommodate heterogeneous medical fine-tuning tasks.
Furthermore, we conceive a dynamic cross-modal prompt optimizing strategy that
optimizes the prompts within the shareable space for implicitly processing the
shareable clinic knowledge. UniDCP is the first Med-VLP model capable of
performing all 8 medical uni-modal and cross-modal tasks over 14 corresponding
datasets, consistently yielding superior results over diverse state-of-the-art
methods.