A step towards understanding why classification helps regression
Silvia L. Pintea Yancong Lin Jouke Dijkstra Jan C. van Gemert
A number of computer vision deep regression approaches report improved
results when adding a classification loss to the regression loss. Here, we
explore why this is useful in practice and when it is beneficial. To do so, we
start from precisely controlled dataset variations and data samplings and find
that the effect of adding a classification loss is the most pronounced for
regression with imbalanced data. We explain these empirical findings by
formalizing the relation between the balanced and imbalanced regression losses.
Finally, we show that our findings hold on two real imbalanced image datasets
for depth estimation (NYUD2-DIR), and age estimation (IMDB-WIKI-DIR), and on
the problem of imbalanced video progress prediction (Breakfast). Our main
takeaway is: for a regression task, if the data sampling is imbalanced, then
add a classification loss.