Multicalibration for Confidence Scoring in LLMs
Gianluca Detommaso Martin Bertran Riccardo Fogliato Aaron Roth
This paper proposes the use of "multicalibration" to yield interpretable and
reliable confidence scores for outputs generated by large language models
(LLMs). Multicalibration asks for calibration not just marginally, but
simultaneously across various intersecting groupings of the data. We show how
to form groupings for prompt/completion pairs that are correlated with the
probability of correctness via two techniques: clustering within an embedding
space, and "self-annotation" - querying the LLM by asking it various yes-or-no
questions about the prompt. We also develop novel variants of multicalibration
algorithms that offer performance improvements by reducing their tendency to
overfit. Through systematic benchmarking across various question answering
datasets and LLMs, we show how our techniques can yield confidence scores that
provide substantial improvements in fine-grained measures of both calibration
and accuracy compared to existing methods.