Aggregating Correlated Estimations with (Almost) no Training
Theo Delemazure François Durand Fabien Mathieu
Many decision problems cannot be solved exactly and use several estimation
algorithms that assign scores to the different available options. The
estimation errors can have various correlations, from low (e.g. between two
very different approaches) to high (e.g. when using a given algorithm with
different hyperparameters). Most aggregation rules would suffer from this
diversity of correlations. In this article, we propose different aggregation
rules that take correlations into account, and we compare them to naive rules
in various experiments based on synthetic data. Our results show that when
sufficient information is known about the correlations between errors, a
maximum likelihood aggregation should be preferred. Otherwise, typically with
limited training data, we recommend a method that we call Embedded Voting (EV).