Federated Offline Policy Learning with Heterogeneous Observational Data
Aldo Gael Carranza Susan Athey
We consider the problem of learning personalized decision policies on
observational data from heterogeneous data sources. Moreover, we examine this
problem in the federated setting where a central server aims to learn a policy
on the data distributed across the heterogeneous sources without exchanging
their raw data. We present a federated policy learning algorithm based on
aggregation of local policies trained with doubly robust offline policy
evaluation and learning strategies. We provide a novel regret analysis for our
approach that establishes a finite-sample upper bound on a notion of global
regret across a distribution of clients. In addition, for any individual
client, we establish a corresponding local regret upper bound characterized by
the presence of distribution shift relative to all other clients. We support
our theoretical findings with experimental results. Our analysis and
experiments provide insights into the value of heterogeneous client
participation in federation for policy learning in heterogeneous settings.