Robust Multi-Modal Density Estimation
Anna Mészáros Julian F. Schumann Javier Alonso-Mora Arkady Zgonnikov Jens Kober
Development of multi-modal, probabilistic prediction models has lead to a
need for comprehensive evaluation metrics. While several metrics can
characterize the accuracy of machine-learned models (e.g., negative
log-likelihood, Jensen-Shannon divergence), these metrics typically operate on
probability densities. Applying them to purely sample-based prediction models
thus requires that the underlying density function is estimated. However,
common methods such as kernel density estimation (KDE) have been demonstrated
to lack robustness, while more complex methods have not been evaluated in
multi-modal estimation problems. In this paper, we present ROME (RObust
Multi-modal density Estimator), a non-parametric approach for density
estimation which addresses the challenge of estimating multi-modal, non-normal,
and highly correlated distributions. ROME utilizes clustering to segment a
multi-modal set of samples into multiple uni-modal ones and then combines
simple KDE estimates obtained for individual clusters in a single multi-modal
estimate. We compared our approach to state-of-the-art methods for density
estimation as well as ablations of ROME, showing that it not only outperforms
established methods but is also more robust to a variety of distributions. Our
results demonstrate that ROME can overcome the issues of over-fitting and
over-smoothing exhibited by other estimators, promising a more robust
evaluation of probabilistic machine learning models.