DPF-Nutrition: Food Nutrition Estimation via Depth Prediction and Fusion
Yuzhe Han Qimin Cheng Wenjin Wu Ziyang Huang
A reasonable and balanced diet is essential for maintaining good health. With
the advancements in deep learning, automated nutrition estimation method based
on food images offers a promising solution for monitoring daily nutritional
intake and promoting dietary health. While monocular image-based nutrition
estimation is convenient, efficient, and economical, the challenge of limited
accuracy remains a significant concern. To tackle this issue, we proposed
DPF-Nutrition, an end-to-end nutrition estimation method using monocular
images. In DPF-Nutrition, we introduced a depth prediction module to generate
depth maps, thereby improving the accuracy of food portion estimation.
Additionally, we designed an RGB-D fusion module that combined monocular images
with the predicted depth information, resulting in better performance for
nutrition estimation. To the best of our knowledge, this was the pioneering
effort that integrated depth prediction and RGB-D fusion techniques in food
nutrition estimation. Comprehensive experiments performed on Nutrition5k
evaluated the effectiveness and efficiency of DPF-Nutrition.