Faithful Model Evaluation for Model-Based Metrics
Palash Goyal Qian Hu Rahul Gupta
Statistical significance testing is used in natural language processing (NLP)
to determine whether the results of a study or experiment are likely to be due
to chance or if they reflect a genuine relationship. A key step in significance
testing is the estimation of confidence interval which is a function of sample
variance. Sample variance calculation is straightforward when evaluating
against ground truth. However, in many cases, a metric model is often used for
evaluation. For example, to compare toxicity of two large language models, a
toxicity classifier is used for evaluation. Existing works usually do not
consider the variance change due to metric model errors, which can lead to
wrong conclusions. In this work, we establish the mathematical foundation of
significance testing for model-based metrics. With experiments on public
benchmark datasets and a production system, we show that considering metric
model errors to calculate sample variances for model-based metrics changes the
conclusions in certain experiments.