Policy Finetuning in Reinforcement Learning via Design of Experiments
  using Offline Data
Ruiqi Zhang Andrea Zanette
In some applications of reinforcement learning, a dataset of pre-collected
experience is already available but it is also possible to acquire some
additional online data to help improve the quality of the policy. However, it
may be preferable to gather additional data with a single, non-reactive
exploration policy and avoid the engineering costs associated with switching
policies.
  In this paper we propose an algorithm with provable guarantees that can
leverage an offline dataset to design a single non-reactive policy for
exploration. We theoretically analyze the algorithm and measure the quality of
the final policy as a function of the local coverage of the original dataset
and the amount of additional data collected.