Combating the effects of speed and delays in end-to-end self-driving
Ardi Tampuu Ilmar Uduste Kristjan Roosild
In the behavioral cloning approach to end-to-end driving, a dataset of expert
driving is collected and the model learns to guess what the expert would do in
different situations. Situations are summarized in observations and the outputs
are low or mid-level commands (e.g. brake, throttle, and steering; or
trajectories). The models learn to match observations at time T to actions
recorded at T or as simultaneously as possible. However, when deploying the
models to the real world (or to an asynchronous simulation), the action
predicted based on observations at time T gets applied at T + $\Delta$ T. In a
variety of cases, $\Delta$ T can be considerable and significantly influence
performance.
  We first demonstrate that driving at two different speeds is effectively two
different tasks. Delays partially cause this difference and linearly amplify
it. Even without computational delays, actuator delays and slipping due to
inertia result in the need to perform actions preemptively when driving fast.
The function mapping observations to commands becomes different compared to
slow driving. We experimentally show that models trained to drive fast cannot
perform the seemingly easier task of driving slow and vice-versa. Good driving
models may be judged to be poor due to testing them at "a safe low speed", a
task they cannot perform.
  Secondly, we show how to counteract the effect of delays in end-to-end
networks by changing the target labels. This is in contrast to the approaches
attempting to minimize the delays, i.e. the cause, not the effect. To exemplify
the problems and solutions in the real world, we use 1:10 scale minicars with
limited computing power, using behavioral cloning for end-to-end driving. Some
of the ideas discussed here may be transferable to the wider context of
self-driving, to vehicles with more compute power and end-to-mid or modular
approaches.