Uncertainty quantification in automated valuation models with locally
  weighted conformal prediction
Anders Hjort Gudmund Horn Hermansen Johan Pensar Jonathan P. Williams
Non-parametric machine learning models, such as random forests and gradient
boosted trees, are frequently used to estimate house prices due to their
predictive accuracy, but such methods are often limited in their ability to
quantify prediction uncertainty. Conformal Prediction (CP) is a model-agnostic
framework for constructing confidence sets around machine learning prediction
models with minimal assumptions. However, due to the spatial dependencies
observed in house prices, direct application of CP leads to confidence sets
that are not calibrated everywhere, i.e., too large of confidence sets in
certain geographical regions and too small in others. We survey various
approaches to adjust the CP confidence set to account for this and demonstrate
their performance on a data set from the housing market in Oslo, Norway. Our
findings indicate that calibrating the confidence sets on a \textit{locally
weighted} version of the non-conformity scores makes the coverage more
consistently calibrated in different geographical regions. We also perform a
simulation study on synthetically generated sale prices to empirically explore
the performance of CP on housing market data under idealized conditions with
known data-generating mechanisms.