MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous
  Driving
Zirui Wu Tianyu Liu Liyi Luo Zhide Zhong Jianteng Chen Hongmin Xiao Chao Hou Haozhe Lou Yuantao Chen Runyi Yang Yuxin Huang Xiaoyu Ye Zike Yan Yongliang Shi Yiyi Liao Hao Zhao
Nowadays, autonomous cars can drive smoothly in ordinary cases, and it is
widely recognized that realistic sensor simulation will play a critical role in
solving remaining corner cases by simulating them. To this end, we propose an
autonomous driving simulator based upon neural radiance fields (NeRFs).
Compared with existing works, ours has three notable features: (1)
Instance-aware. Our simulator models the foreground instances and background
environments separately with independent networks so that the static (e.g.,
size and appearance) and dynamic (e.g., trajectory) properties of instances can
be controlled separately. (2) Modular. Our simulator allows flexible switching
between different modern NeRF-related backbones, sampling strategies, input
modalities, etc. We expect this modular design to boost academic progress and
industrial deployment of NeRF-based autonomous driving simulation. (3)
Realistic. Our simulator set new state-of-the-art photo-realism results given
the best module selection. Our simulator will be open-sourced while most of our
counterparts are not. Project page: https://open-air-sun.github.io/mars/.