Using Images as Covariates: Measuring Curb Appeal with Deep Learning
Ardyn Nordstrom Morgan Nordstrom Matthew D. Webb
This paper details an innovative methodology to integrate image data into
traditional econometric models. Motivated by forecasting sales prices for
residential real estate, we harness the power of deep learning to add
"information" contained in images as covariates. Specifically, images of homes
were categorized and encoded using an ensemble of image classifiers (ResNet-50,
VGG16, MobileNet, and Inception V3). Unique features presented within each
image were further encoded through panoptic segmentation. Forecasts from a
neural network trained on the encoded data results in improved out-of-sample
predictive power. We also combine these image-based forecasts with standard
hedonic real estate property and location characteristics, resulting in a
unified dataset. We show that image-based forecasts increase the accuracy of
hedonic forecasts when encoded features are regarded as additional covariates.
We also attempt to "explain" which covariates the image-based forecasts are
most highly correlated with. The study exemplifies the benefits of
interdisciplinary methodologies, merging machine learning and econometrics to
harness untapped data sources for more accurate forecasting.