Streaming Motion Forecasting for Autonomous Driving
Ziqi Pang Deva Ramanan Mengtian Li Yu-Xiong Wang
Trajectory forecasting is a widely-studied problem for autonomous navigation.
However, existing benchmarks evaluate forecasting based on independent
snapshots of trajectories, which are not representative of real-world
applications that operate on a continuous stream of data. To bridge this gap,
we introduce a benchmark that continuously queries future trajectories on
streaming data and we refer to it as "streaming forecasting." Our benchmark
inherently captures the disappearance and re-appearance of agents, presenting
the emergent challenge of forecasting for occluded agents, which is a
safety-critical problem yet overlooked by snapshot-based benchmarks. Moreover,
forecasting in the context of continuous timestamps naturally asks for temporal
coherence between predictions from adjacent timestamps. Based on this
benchmark, we further provide solutions and analysis for streaming forecasting.
We propose a plug-and-play meta-algorithm called "Predictive Streamer" that can
adapt any snapshot-based forecaster into a streaming forecaster. Our algorithm
estimates the states of occluded agents by propagating their positions with
multi-modal trajectories, and leverages differentiable filters to ensure
temporal consistency. Both occlusion reasoning and temporal coherence
strategies significantly improve forecasting quality, resulting in 25% smaller
endpoint errors for occluded agents and 10-20% smaller fluctuations of
trajectories. Our work is intended to generate interest within the community by
highlighting the importance of addressing motion forecasting in its intrinsic
streaming setting. Code is available at
https://github.com/ziqipang/StreamingForecasting.