Food Image Classification and Segmentation with Attention-based Multiple
  Instance Learning
Valasia Vlachopoulou Ioannis Sarafis Alexandros Papadopoulos
The demand for accurate food quantification has increased in the recent
years, driven by the needs of applications in dietary monitoring. At the same
time, computer vision approaches have exhibited great potential in automating
tasks within the food domain. Traditionally, the development of machine
learning models for these problems relies on training data sets with
pixel-level class annotations. However, this approach introduces challenges
arising from data collection and ground truth generation that quickly become
costly and error-prone since they must be performed in multiple settings and
for thousands of classes. To overcome these challenges, the paper presents a
weakly supervised methodology for training food image classification and
semantic segmentation models without relying on pixel-level annotations. The
proposed methodology is based on a multiple instance learning approach in
combination with an attention-based mechanism. At test time, the models are
used for classification and, concurrently, the attention mechanism generates
semantic heat maps which are used for food class segmentation. In the paper, we
conduct experiments on two meta-classes within the FoodSeg103 data set to
verify the feasibility of the proposed approach and we explore the functioning
properties of the attention mechanism.