Diffusion Model is a Good Pose Estimator from 3D RF-Vision
Junqiao Fan Jianfei Yang Yuecong Xu Lihua Xie
Human pose estimation (HPE) from Radio Frequency vision (RF-vision) performs
human sensing using RF signals that penetrate obstacles without revealing
privacy (e.g., facial information). Recently, mmWave radar has emerged as a
promising RF-vision sensor, providing radar point clouds by processing RF
signals. However, the mmWave radar has a limited resolution with severe noise,
leading to inaccurate and inconsistent human pose estimation. This work
proposes mmDiff, a novel diffusion-based pose estimator tailored for noisy
radar data. Our approach aims to provide reliable guidance as conditions to
diffusion models. Two key challenges are addressed by mmDiff: (1)
miss-detection of parts of human bodies, which is addressed by a module that
isolates feature extraction from different body parts, and (2) signal
inconsistency due to environmental interference, which is tackled by
incorporating prior knowledge of body structure and motion. Several modules are
designed to achieve these goals, whose features work as the conditions for the
subsequent diffusion model, eliminating the miss-detection and instability of
HPE based on RF-vision. Extensive experiments demonstrate that mmDiff
outperforms existing methods significantly, achieving state-of-the-art
performances on public datasets.