An Empirical Study of the Generalization Ability of Lidar 3D Object
  Detectors to Unseen Domains
George Eskandar Chongzhe Zhang Abhishek Kaushik Karim Guirguis Mohamed Sayed Bin Yang
3D Object Detectors (3D-OD) are crucial for understanding the environment in
many robotic tasks, especially autonomous driving. Including 3D information via
Lidar sensors improves accuracy greatly. However, such detectors perform poorly
on domains they were not trained on, i.e. different locations, sensors,
weather, etc., limiting their reliability in safety-critical applications.
There exist methods to adapt 3D-ODs to these domains; however, these methods
treat 3D-ODs as a black box, neglecting underlying architectural decisions and
source-domain training strategies. Instead, we dive deep into the details of
3D-ODs, focusing our efforts on fundamental factors that influence robustness
prior to domain adaptation.
  We systematically investigate four design choices (and the interplay between
them) often overlooked in 3D-OD robustness and domain adaptation: architecture,
voxel encoding, data augmentations, and anchor strategies. We assess their
impact on the robustness of nine state-of-the-art 3D-ODs across six benchmarks
encompassing three types of domain gaps - sensor type, weather, and location.
  Our main findings are: (1) transformer backbones with local point features
are more robust than 3D CNNs, (2) test-time anchor size adjustment is crucial
for adaptation across geographical locations, significantly boosting scores
without retraining, (3) source-domain augmentations allow the model to
generalize to low-resolution sensors, and (4) surprisingly, robustness to bad
weather is improved when training directly on more clean weather data than on
training with bad weather data. We outline our main conclusions and findings to
provide practical guidance on developing more robust 3D-ODs.