MMBaT: A Multi-task Framework for mmWave-based Human Body Reconstruction
  and Translation Prediction
Jiarui Yang Songpengcheng Xia Yifan Song Qi Wu Ling Pei
Human body reconstruction with Millimeter Wave (mmWave) radar point clouds
has gained significant interest due to its ability to work in adverse
environments and its capacity to mitigate privacy concerns associated with
traditional camera-based solutions. Despite pioneering efforts in this field,
two challenges persist. Firstly, raw point clouds contain massive noise points,
usually caused by the ambient objects and multi-path effects of Radio Frequency
(RF) signals. Recent approaches typically rely on prior knowledge or elaborate
preprocessing methods, limiting their applicability. Secondly, even after noise
removal, the sparse and inconsistent body-related points pose an obstacle to
accurate human body reconstruction. To address these challenges, we introduce
mmBaT, a novel multi-task deep learning framework that concurrently estimates
the human body and predicts body translations in subsequent frames to extract
body-related point clouds. Our method is evaluated on two public datasets that
are collected with different radar devices and noise levels. A comprehensive
comparison against other state-of-the-art methods demonstrates our method has a
superior reconstruction performance and generalization ability from noisy raw
data, even when compared to methods provided with body-related point clouds.