Invariant Subspace Decomposition
Margherita Lazzaretto Jonas Peters Niklas Pfister
We consider the task of predicting a response Y from a set of covariates X in
settings where the conditional distribution of Y given X changes over time. For
this to be feasible, assumptions on how the conditional distribution changes
over time are required. Existing approaches assume, for example, that changes
occur smoothly over time so that short-term prediction using only the recent
past becomes feasible. In this work, we propose a novel invariance-based
framework for linear conditionals, called Invariant Subspace Decomposition
(ISD), that splits the conditional distribution into a time-invariant and a
residual time-dependent component. As we show, this decomposition can be
utilized both for zero-shot and time-adaptation prediction tasks, that is,
settings where either no or a small amount of training data is available at the
time points we want to predict Y at, respectively. We propose a practical
estimation procedure, which automatically infers the decomposition using tools
from approximate joint matrix diagonalization. Furthermore, we provide finite
sample guarantees for the proposed estimator and demonstrate empirically that
it indeed improves on approaches that do not use the additional invariant
structure.