Think Twice Before Assure: Confidence Estimation for Large Language
  Models through Reflection on Multiple Answers
Moxin Li Wenjie Wang Fuli Feng Fengbin Zhu Qifan Wang Tat-Seng Chua
Confidence estimation aiming to evaluate output trustability is crucial for
the application of large language models (LLM), especially the black-box ones.
Existing confidence estimation of LLM is typically not calibrated due to the
overconfidence of LLM on its generated incorrect answers. Existing approaches
addressing the overconfidence issue are hindered by a significant limitation
that they merely consider the confidence of one answer generated by LLM. To
tackle this limitation, we propose a novel paradigm that thoroughly evaluates
the trustability of multiple candidate answers to mitigate the overconfidence
on incorrect answers. Building upon this paradigm, we introduce a two-step
framework, which firstly instructs LLM to reflect and provide justifications
for each answer, and then aggregates the justifications for comprehensive
confidence estimation. This framework can be integrated with existing
confidence estimation approaches for superior calibration. Experimental results
on six datasets of three tasks demonstrate the rationality and effectiveness of
the proposed framework.