KARNet: Kalman Filter Augmented Recurrent Neural Network for Learning
  World Models in Autonomous Driving Tasks
Hemanth Manjunatha Andrey Pak Dimitar Filev Panagiotis Tsiotras
Autonomous driving has received a great deal of attention in the automotive
industry and is often seen as the future of transportation. The development of
autonomous driving technology has been greatly accelerated by the growth of
end-to-end machine learning techniques that have been successfully used for
perception, planning, and control tasks. An important aspect of autonomous
driving planning is knowing how the environment evolves in the immediate future
and taking appropriate actions. An autonomous driving system should effectively
use the information collected from the various sensors to form an abstract
representation of the world to maintain situational awareness. For this
purpose, deep learning models can be used to learn compact latent
representations from a stream of incoming data. However, most deep learning
models are trained end-to-end and do not incorporate any prior knowledge (e.g.,
from physics) of the vehicle in the architecture. In this direction, many works
have explored physics-infused neural network (PINN) architectures to infuse
physics models during training. Inspired by this observation, we present a
Kalman filter augmented recurrent neural network architecture to learn the
latent representation of the traffic flow using front camera images only. We
demonstrate the efficacy of the proposed model in both imitation and
reinforcement learning settings using both simulated and real-world datasets.
The results show that incorporating an explicit model of the vehicle (states
estimated using Kalman filtering) in the end-to-end learning significantly
increases performance.