R-Tuning: Teaching Large Language Models to Refuse Unknown Questions
Hanning Zhang Shizhe Diao Yong Lin Yi R. Fung Qing Lian Xingyao Wang Yangyi Chen Heng Ji Tong Zhang
Large language models (LLMs) have revolutionized numerous domains with their
impressive performance but still face their challenges. A predominant issue is
the propensity for these models to generate non-existent facts, a concern
termed hallucination. Our research is motivated by the observation that
previous instruction tuning methods force the model to complete a sentence no
matter whether the model knows the knowledge or not. When the question is out
of the parametric knowledge, it will try to make up something and fail to
indicate when it lacks knowledge. In this paper, we present a new approach
called Refusal-Aware Instruction Tuning (R-Tuning). This approach is formalized
by first identifying the knowledge gap between parametric knowledge and the
instruction tuning data. Then, we construct the refusal-aware data based on the
knowledge intersection, to tune LLMs to refrain from responding to questions
beyond its parametric knowledge. Experimental results demonstrate this new
instruction tuning approach effectively improves a model's ability to answer
known questions and refrain from answering unknown questions. Furthermore, when
tested on out-of-domain datasets, the refusal ability was found to be a
meta-skill that could be generalized to other tasks. Further analysis
surprisingly finds that learning the uncertainty during training displays a
better ability to estimate uncertainty than uncertainty-based testing. Our code
will be released at https://github.com/shizhediao/R-Tuning.