Estimation of Models with Limited Data by Leveraging Shared Structure
Maryann Rui Thibaut Horel Munther Dahleh
Modern data sets, such as those in healthcare and e-commerce, are often
derived from many individuals or systems but have insufficient data from each
source alone to separately estimate individual, often high-dimensional, model
parameters. If there is shared structure among systems however, it may be
possible to leverage data from other systems to help estimate individual
parameters, which could otherwise be non-identifiable. In this paper, we assume
systems share a latent low-dimensional parameter space and propose a method for
recovering $d$-dimensional parameters for $N$ different linear systems, even
when there are only $T<d$ observations per system. To do so, we develop a
three-step algorithm which estimates the low-dimensional subspace spanned by
the systems' parameters and produces refined parameter estimates within the
subspace. We provide finite sample subspace estimation error guarantees for our
proposed method. Finally, we experimentally validate our method on simulations
with i.i.d. regression data and as well as correlated time series data.