Differentially Private Model-Based Offline Reinforcement Learning
Alexandre Rio Merwan Barlier Igor Colin Albert Thomas
We address offline reinforcement learning with privacy guarantees, where the
goal is to train a policy that is differentially private with respect to
individual trajectories in the dataset. To achieve this, we introduce DP-MORL,
an MBRL algorithm coming with differential privacy guarantees. A private model
of the environment is first learned from offline data using DP-FedAvg, a
training method for neural networks that provides differential privacy
guarantees at the trajectory level. Then, we use model-based policy
optimization to derive a policy from the (penalized) private model, without any
further interaction with the system or access to the input data. We empirically
show that DP-MORL enables the training of private RL agents from offline data
and we furthermore outline the price of privacy in this setting.