Your Finetuned Large Language Model is Already a Powerful
  Out-of-distribution Detector
Andi Zhang Tim Z. Xiao Weiyang Liu Robert Bamler Damon Wischik
We revisit the likelihood ratio between a pretrained large language model
(LLM) and its finetuned variant as a criterion for out-of-distribution (OOD)
detection. The intuition behind such a criterion is that, the pretrained LLM
has the prior knowledge about OOD data due to its large amount of training
data, and once finetuned with the in-distribution data, the LLM has sufficient
knowledge to distinguish their difference. Leveraging the power of LLMs, we
show that, for the first time, the likelihood ratio can serve as an effective
OOD detector. Moreover, we apply the proposed LLM-based likelihood ratio to
detect OOD questions in question-answering (QA) systems, which can be used to
improve the performance of specialized LLMs for general questions. Given that
likelihood can be easily obtained by the loss functions within contemporary
neural network frameworks, it is straightforward to implement this approach in
practice. Since both the pretrained LLMs and its various finetuned models are
available, our proposed criterion can be effortlessly incorporated for OOD
detection without the need for further training. We conduct comprehensive
evaluation across on multiple settings, including far OOD, near OOD, spam
detection, and QA scenarios, to demonstrate the effectiveness of the method.