H2RSVLM: Towards Helpful and Honest Remote Sensing Large Vision Language
  Model
Chao Pang Jiang Wu Jiayu Li Yi Liu Jiaxing Sun Weijia Li Xingxing Weng Shuai Wang Litong Feng Gui-Song Xia Conghui He
The generic large Vision-Language Models (VLMs) is rapidly developing, but
still perform poorly in Remote Sensing (RS) domain, which is due to the unique
and specialized nature of RS imagery and the comparatively limited spatial
perception of current VLMs. Existing Remote Sensing specific Vision Language
Models (RSVLMs) still have considerable potential for improvement, primarily
owing to the lack of large-scale, high-quality RS vision-language datasets. We
constructed HqDC-1.4M, the large scale High quality and Detailed Captions for
RS images, containing 1.4 million image-caption pairs, which not only enhance
the RSVLM's understanding of RS images but also significantly improve the
model's spatial perception abilities, such as localization and counting,
thereby increasing the helpfulness of the RSVLM. Moreover, to address the
inevitable "hallucination" problem in RSVLM, we developed RSSA, the first
dataset aimed at enhancing the Self-Awareness capability of RSVLMs. By
incorporating a variety of unanswerable questions into typical RS visual
question-answering tasks, RSSA effectively improves the truthfulness and
reduces the hallucinations of the model's outputs, thereby enhancing the
honesty of the RSVLM. Based on these datasets, we proposed the H2RSVLM, the
Helpful and Honest Remote Sensing Vision Language Model. H2RSVLM has achieved
outstanding performance on multiple RS public datasets and is capable of
recognizing and refusing to answer the unanswerable questions, effectively
mitigating the incorrect generations. We will release the code, data and model
weights at https://github.com/opendatalab/H2RSVLM .