Long-Tailed Continual Learning For Visual Food Recognition
Jiangpeng He Luotao Lin Jack Ma Heather A. Eicher-Miller Fengqing Zhu
Deep learning based food recognition has achieved remarkable progress in
predicting food types given an eating occasion image. However, there are two
major obstacles that hinder deployment in real world scenario. First, as new
foods appear sequentially overtime, a trained model needs to learn the new
classes continuously without causing catastrophic forgetting for already
learned knowledge of existing food types. Second, the distribution of food
images in real life is usually long-tailed as a small number of popular food
types are consumed more frequently than others, which can vary in different
populations. This requires the food recognition method to learn from
class-imbalanced data by improving the generalization ability on instance-rare
food classes. In this work, we focus on long-tailed continual learning and aim
to address both aforementioned challenges. As existing long-tailed food image
datasets only consider healthy people population, we introduce two new
benchmark food image datasets, VFN-INSULIN and VFN-T2D, which exhibits on the
real world food consumption for insulin takers and individuals with type 2
diabetes without taking insulin, respectively. We propose a novel end-to-end
framework for long-tailed continual learning, which effectively addresses the
catastrophic forgetting by applying an additional predictor for knowledge
distillation to avoid misalignment of representation during continual learning.
We also introduce a novel data augmentation technique by integrating
class-activation-map (CAM) and CutMix, which significantly improves the
generalization ability for instance-rare food classes to address the
class-imbalance issue. The proposed method show promising performance with
large margin improvements compared with existing methods.