Knowledge-tuning Large Language Models with Structured Medical Knowledge
  Bases for Reliable Response Generation in Chinese
Haochun Wang Sendong Zhao Zewen Qiang Zijian Li Nuwa Xi Yanrui Du MuZhen Cai Haoqiang Guo Yuhan Chen Haoming Xu Bing Qin Ting Liu
Large Language Models (LLMs) have demonstrated remarkable success in diverse
natural language processing (NLP) tasks in general domains. However, LLMs
sometimes generate responses with the hallucination about medical facts due to
limited domain knowledge. Such shortcomings pose potential risks in the
utilization of LLMs within medical contexts. To address this challenge, we
propose knowledge-tuning, which leverages structured medical knowledge bases
for the LLMs to grasp domain knowledge efficiently and facilitate reliable
response generation. We also release cMedKnowQA, a Chinese medical knowledge
question-answering dataset constructed from medical knowledge bases to assess
the medical knowledge proficiency of LLMs. Experimental results show that the
LLMs which are knowledge-tuned with cMedKnowQA, can exhibit higher levels of
accuracy in response generation compared with vanilla instruction-tuning and
offer a new reliable way for the domain adaptation of LLMs.