In many robotic applications, it is crucial to maintain a belief about the state of 
 a system, like the location of a robot or the pose of an object.
 These state estimates serve as input for planning and decision making and 
 provide feedback during task execution. 
 Recursive Bayesian Filtering algorithms address the state estimation problem,
 but they require a model of the process dynamics and the sensory observations as well as 
 noise estimates that quantify the accuracy of these models. 
 Recently, multiple works have demonstrated that the process and sensor models can be 
 learned by end-to-end training through differentiable versions of Recursive Filtering methods.
 However, even if the predictive models are known, finding suitable noise models 
 remains challenging. Therefore, many practical applications rely on very simplistic noise 
 models. 
 Our hypothesis is that end-to-end training through differentiable Bayesian 
 Filters enables us to learn more complex heteroscedastic noise models for
 the system dynamics. We evaluate learning such models with different types of 
 filtering algorithms and on two different robotic tasks. Our experiments show that especially 
 for sampling-based filters like the Particle Filter, learning heteroscedastic noise 
 models can drastically improve the tracking performance in comparison to using 
 constant noise models. For many real-world systems that we would like to control, we cannot directly observe the current state directly. However, in order to stabilize a system at a goal state or make it track a trajectory, we need to have access to state feedback. An observer provides an estimate of the current system state from sensor measurements. Recursive Bayesian Filtering is a probabilistic approach towards estimating a belief about the current state. The method relies on a process model that predicts how the system behaves over time and an observation model that generates the expected observations given the predicted state. While the approach itself is general and makes few assumptions, the challenge is to formulate the process and observation models and to estimate the noise in these models. Process and observation noise quantify how certain the filter is about either the prediction or the observations. This information is used to determine how much the predicted state is updated based on the observation.Deep neural networks are well suited for tasks that require finding patterns or extracting information from raw, high-dimensional input signals and compressing them into a more compact representation. They have therefore become the method of choice especially in perception problems. For many robotics tasks like modeling dynamics, planning or tracking however, it has been shown that combining prior knowledge in the form of analytical models and/or algorithmic structure with trainable network components leads to better performance and generalizability than trying to learn the complete tasks from scratch BID17 BID11 BID9 BID23 BID19 BID8 BID6 BID12 .Specifically , BID8 BID6 BID9 BID12 have presented differentiable Bayesian Filtering algorithms. The authors focus on learning the observation and dynamics models end-to-end through the filters and demonstrate that the recursive filtering structure improves prediction results over using recurrent neural networks that were trained for the same task.In many robotic applications, it is possible to formulate the process and observation model based on first-order principles. However, finding appropriate values for the process and observation noise is often difficult and despite of much research on identification methods (e.g. BID2 BID25 ) they are often tuned manually. To reduce the tedious tuning effort, the noise models are typically assumed to be a Gaussian with zero mean and constant covariance. Many real systems can however be better modeled with heteroscedastic noise models, where the level of uncertainty depends on the state of the system and/or possible control inputs. Taking heterostochasticity into account has been demonstrated to improve filtering performance in many robotic tasks BID1 BID14 .In this work, we propose a method to learn heteroscedastic noise models from data by optimizing the prediction likelihood end-to-end through differentiable Bayesian Filters. In addition to differentiable Extended Kalman Filters and Particle Filters, which have been proposed in related work, we also propose two different versions of the Unscented Kalman Filter.In our experiments we focus on learning the noise models and therefore assume that observation and process models are known or at least pretrained. We evaluate the performance of the different filters and noise models on two different real-world robotic problems: (i) Visual Odometry for an driving car BID6 BID9 BID4 which has simple smooth dynamics and a low-dimensional state, and (ii) Visual tracking of an object that is pushed by a robot (Yu et al., 2016; BID17 . Planar pushing has challenging, discontinuous dynamics and was shown to have a heteroscedastic noise distribution BID1 . Furthermore, the dimensionality of the state is double of the Visual Odometry task.Our experiments show that using heteroscedastic process noise models drastically improves the tracking performance of the Particle Filter and Unscented Filter variants and facilitated learning as compared to learning a constant process noise model. While learning the noise models can be beneficial for all filters, the tracking performance of the EKF turned out to be least sensitive to the noise models. In comparison to the process noise, learning the observation noise did not improve the results much for the two tasks we evaluated. We proposed to optimize the process and observation noise for Bayesian Filters through end-to-end training and evaluated the method with different filtering algorithms and on two robotic applications. Our experiments showed that learning the process noise is especially important for filters that sample around the mean estimate of the state, like the Particle Filter but also the Unscented Kalman Filters. The Extended Kalman Filter in contrast proved to be most robust to suboptimal choices of the noise models. While this makes it a good choice for problems with simple and smooth dynamics, our experiments on the pushing task demonstrated that the (optimized) Unscented Filters can perform better on problems with more complex and even discontinuous dynamics.Training a state-dependent process noise model instead of a constant one improves the prediction accuracy for dynamic systems that are expected to have heteroscedastic noise. In our experiments, it also facilitated learning in general and lead to faster convergence of the models.We also used a heteroscedastic observation noise model in all our experiments. But different from the results in BID6 , we could not see a large benefit from it: Inspection on the pushing task showed that larger errors in the prediction of the preprocessing networks were not associated with higher observation noise. Identifying inputs that will lead to bad predictions is a difficult task if no obvious problems like occlusions are present to explain such outliers. Developing better methods for communicating uncertainty about the predictions of a neural network would thus be an impotent next step to further improve the performance of differentiable Bayesian Filters. The basic steps of the Extended Kalman Filter can be directly implemented in Tensorflow without any modifications. The only aspect of interest is how to compute the Jacobians of the process and observation model. Tensorflow implements auto differentiation, but has (as of now) no native support for computing Jacobians. While it can be done, it requires looping over the dimensions of the differentiated variable one by one, which we found to be relatively slow, especially during graph-construction. We therefore recommend to manually derive the Jacobians where applicable.
We evaluate learning heteroscedastic noise models within different Differentiable Bayes Filters Proposes to learn heteroscedastic noise models from data by optimizing the prediction likelihood end-toend through differentiable Bayesian Filters and two different versions of the Unscented Kalman Filter Revisits Bayes filters and evaluates the benefit of training the observation and process noise models while keeping all other models fixed This paper presents a method to learn and use state and observation dependent noise in traditional Bayesian filtering algorithms. The approach consists of constructing a neural network model which takes as input the raw observation data and produces a compact representation and an associated diagonal covariance.