ReasonNet: End-to-End Driving with Temporal and Global Reasoning
Hao Shao Letian Wang Ruobing Chen Steven L. Waslander Hongsheng Li Yu Liu
The large-scale deployment of autonomous vehicles is yet to come, and one of
the major remaining challenges lies in urban dense traffic scenarios. In such
cases, it remains challenging to predict the future evolution of the scene and
future behaviors of objects, and to deal with rare adverse events such as the
sudden appearance of occluded objects. In this paper, we present ReasonNet, a
novel end-to-end driving framework that extensively exploits both temporal and
global information of the driving scene. By reasoning on the temporal behavior
of objects, our method can effectively process the interactions and
relationships among features in different frames. Reasoning about the global
information of the scene can also improve overall perception performance and
benefit the detection of adverse events, especially the anticipation of
potential danger from occluded objects. For comprehensive evaluation on
occlusion events, we also release publicly a driving simulation benchmark
DriveOcclusionSim consisting of diverse occlusion events. We conduct extensive
experiments on multiple CARLA benchmarks, where our model outperforms all prior
methods, ranking first on the sensor track of the public CARLA Leaderboard.