Density-Regression: Efficient and Distance-Aware Deep Regressor for
  Uncertainty Estimation under Distribution Shifts
Ha Manh Bui Anqi Liu
Morden deep ensembles technique achieves strong uncertainty estimation
performance by going through multiple forward passes with different models.
This is at the price of a high storage space and a slow speed in the inference
(test) time. To address this issue, we propose Density-Regression, a method
that leverages the density function in uncertainty estimation and achieves fast
inference by a single forward pass. We prove it is distance aware on the
feature space, which is a necessary condition for a neural network to produce
high-quality uncertainty estimation under distribution shifts. Empirically, we
conduct experiments on regression tasks with the cubic toy dataset, benchmark
UCI, weather forecast with time series, and depth estimation under real-world
shifted applications. We show that Density-Regression has competitive
uncertainty estimation performance under distribution shifts with modern deep
regressors while using a lower model size and a faster inference speed.