VoxelKP: A Voxel-based Network Architecture for Human Keypoint
  Estimation in LiDAR Data
Jian Shi Peter Wonka
We present \textit{VoxelKP}, a novel fully sparse network architecture
tailored for human keypoint estimation in LiDAR data. The key challenge is that
objects are distributed sparsely in 3D space, while human keypoint detection
requires detailed local information wherever humans are present. We propose
four novel ideas in this paper. First, we propose sparse selective kernels to
capture multi-scale context. Second, we introduce sparse box-attention to focus
on learning spatial correlations between keypoints within each human instance.
Third, we incorporate a spatial encoding to leverage absolute 3D coordinates
when projecting 3D voxels to a 2D grid encoding a bird's eye view. Finally, we
propose hybrid feature learning to combine the processing of per-voxel features
with sparse convolution. We evaluate our method on the Waymo dataset and
achieve an improvement of $27\%$ on the MPJPE metric compared to the
state-of-the-art, \textit{HUM3DIL}, trained on the same data, and $12\%$
against the state-of-the-art, \textit{GC-KPL}, pretrained on a $25\times$
larger dataset. To the best of our knowledge, \textit{VoxelKP} is the first
single-staged, fully sparse network that is specifically designed for
addressing the challenging task of 3D keypoint estimation from LiDAR data,
achieving state-of-the-art performances. Our code is available at
\url{https://github.com/shijianjian/VoxelKP}.