Collaboratively Learning Linear Models with Structured Missing Data
Chen Cheng Gary Cheng John Duchi
We study the problem of collaboratively learning least squares estimates for
$m$ agents. Each agent observes a different subset of the
features$\unicode{x2013}$e.g., containing data collected from sensors of
varying resolution. Our goal is to determine how to coordinate the agents in
order to produce the best estimator for each agent. We propose a distributed,
semi-supervised algorithm Collab, consisting of three steps: local training,
aggregation, and distribution. Our procedure does not require communicating the
labeled data, making it communication efficient and useful in settings where
the labeled data is inaccessible. Despite this handicap, our procedure is
nearly asymptotically local minimax optimal$\unicode{x2013}$even among
estimators allowed to communicate the labeled data such as imputation methods.
We test our method on real and synthetic data.