Mitigating Dialogue Hallucination for Large Multi-modal Models via
  Adversarial Instruction Tuning
Dongmin Park Zhaofang Qian Guangxing Han Ser-Nam Lim
Mitigating hallucinations of Large Multi-modal Models(LMMs) is crucial to
enhance their reliability for general-purpose assistants. This paper shows that
such hallucinations of LMMs can be significantly exacerbated by preceding
user-system dialogues. To precisely measure this, we first present an
evaluation benchmark by extending popular multi-modal benchmark datasets with
prepended hallucinatory dialogues generated by our novel Adversarial Question
Generator, which can automatically generate image-related yet adversarial
dialogues by adopting adversarial attacks on LMMs. On our benchmark, the
zero-shot performance of state-of-the-art LMMs dropped significantly for both
the VQA and Captioning tasks. Next, we further reveal this hallucination is
mainly due to the prediction bias toward preceding dialogues rather than visual
content. To reduce this bias, we propose Adversarial Instruction Tuning that
robustly fine-tunes LMMs on augmented multi-modal instruction-following
datasets with hallucinatory dialogues. Extensive experiments show that our
proposed approach successfully reduces dialogue hallucination while maintaining
or even improving performance.