DCID: Deep Canonical Information Decomposition
Alexander Rakowski Christoph Lippert
We consider the problem of identifying the signal shared between two
one-dimensional target variables, in the presence of additional multivariate
observations. Canonical Correlation Analysis (CCA)-based methods have
traditionally been used to identify shared variables, however, they were
designed for multivariate targets and only offer trivial solutions for
univariate cases. In the context of Multi-Task Learning (MTL), various models
were postulated to learn features that are sparse and shared across multiple
tasks. However, these methods were typically evaluated by their predictive
performance. To the best of our knowledge, no prior studies systematically
evaluated models in terms of correctly recovering the shared signal. Here, we
formalize the setting of univariate shared information retrieval, and propose
ICM, an evaluation metric which can be used in the presence of ground-truth
labels, quantifying 3 aspects of the learned shared features. We further
propose Deep Canonical Information Decomposition (DCID) - a simple, yet
effective approach for learning the shared variables. We benchmark the models
on a range of scenarios on synthetic data with known ground-truths and observe
DCID outperforming the baselines in a wide range of settings. Finally, we
demonstrate a real-life application of DCID on brain Magnetic Resonance Imaging
(MRI) data, where we are able to extract more accurate predictors of changes in
brain regions and obesity. The code for our experiments as well as the
supplementary materials are available at https://github.com/alexrakowski/dcid