SLEDGE: Synthesizing Simulation Environments for Driving Agents with
  Generative Models
Kashyap Chitta Daniel Dauner Andreas Geiger
SLEDGE is the first generative simulator for vehicle motion planning trained
on real-world driving logs. Its core component is a learned model that is able
to generate agent bounding boxes and lane graphs. The model's outputs serve as
an initial state for traffic simulation. The unique properties of the entities
to be generated for SLEDGE, such as their connectivity and variable count per
scene, render the naive application of most modern generative models to this
task non-trivial. Therefore, together with a systematic study of existing lane
graph representations, we introduce a novel raster-to-vector autoencoder
(RVAE). It encodes agents and the lane graph into distinct channels in a
rasterized latent map. This facilitates both lane-conditioned agent generation
and combined generation of lanes and agents with a Diffusion Transformer. Using
generated entities in SLEDGE enables greater control over the simulation, e.g.
upsampling turns or increasing traffic density. Further, SLEDGE can support
500m long routes, a capability not found in existing data-driven simulators
like nuPlan. It presents new challenges for planning algorithms, evidenced by
failure rates of over 40% for PDM, the winner of the 2023 nuPlan challenge,
when tested on hard routes and dense traffic generated by our model. Compared
to nuPlan, SLEDGE requires 500$\times$ less storage to set up (<4GB), making it
a more accessible option and helping with democratizing future research in this
field.