Advancing Smart Malnutrition Monitoring: A Multi-Modal Learning Approach
  for Vital Health Parameter Estimation
Ashish Marisetty Prathistith Raj M Praneeth Nemani Venkanna Udutalapally Debanjan Das
Malnutrition poses a significant threat to global health, resulting from an
inadequate intake of essential nutrients that adversely impacts vital organs
and overall bodily functioning. Periodic examinations and mass screenings,
incorporating both conventional and non-invasive techniques, have been employed
to combat this challenge. However, these approaches suffer from critical
limitations, such as the need for additional equipment, lack of comprehensive
feature representation, absence of suitable health indicators, and the
unavailability of smartphone implementations for precise estimations of Body
Fat Percentage (BFP), Basal Metabolic Rate (BMR), and Body Mass Index (BMI) to
enable efficient smart-malnutrition monitoring. To address these constraints,
this study presents a groundbreaking, scalable, and robust smart
malnutrition-monitoring system that leverages a single full-body image of an
individual to estimate height, weight, and other crucial health parameters
within a multi-modal learning framework. Our proposed methodology involves the
reconstruction of a highly precise 3D point cloud, from which 512-dimensional
feature embeddings are extracted using a headless-3D classification network.
Concurrently, facial and body embeddings are also extracted, and through the
application of learnable parameters, these features are then utilized to
estimate weight accurately. Furthermore, essential health metrics, including
BMR, BFP, and BMI, are computed to conduct a comprehensive analysis of the
subject's health, subsequently facilitating the provision of personalized
nutrition plans. While being robust to a wide range of lighting conditions
across multiple devices, our model achieves a low Mean Absolute Error (MAE) of
$\pm$ 4.7 cm and $\pm$ 5.3 kg in estimating height and weight.