Contrastive inverse regression for dimension reduction
Sam Hawke Hengrui Luo Didong Li
Supervised dimension reduction (SDR) has been a topic of growing interest in
data science, as it enables the reduction of high-dimensional covariates while
preserving the functional relation with certain response variables of interest.
However, existing SDR methods are not suitable for analyzing datasets collected
from case-control studies. In this setting, the goal is to learn and exploit
the low-dimensional structure unique to or enriched by the case group, also
known as the foreground group. While some unsupervised techniques such as the
contrastive latent variable model and its variants have been developed for this
purpose, they fail to preserve the functional relationship between the
dimension-reduced covariates and the response variable. In this paper, we
propose a supervised dimension reduction method called contrastive inverse
regression (CIR) specifically designed for the contrastive setting. CIR
introduces an optimization problem defined on the Stiefel manifold with a
non-standard loss function. We prove the convergence of CIR to a local optimum
using a gradient descent-based algorithm, and our numerical study empirically
demonstrates the improved performance over competing methods for
high-dimensional data.