TIC-TAC: A Framework To Learn And Evaluate Your Covariance
Megh Shukla Mathieu Salzmann Alexandre Alahi
We study the problem of unsupervised heteroscedastic covariance estimation,
where the goal is to learn the multivariate target distribution $\mathcal{N}(y,
\Sigma_y | x )$ given an observation $x$. This problem is particularly
challenging as $\Sigma_{y}$ varies for different samples (heteroscedastic) and
no annotation for the covariance is available (unsupervised). Typically,
state-of-the-art methods predict the mean $f_{\theta}(x)$ and covariance
$\textrm{Cov}(f_{\theta}(x))$ of the target distribution through two neural
networks trained using the negative log-likelihood. This raises two questions:
(1) Does the predicted covariance truly capture the randomness of the predicted
mean? (2) In the absence of ground-truth annotation, how can we quantify the
performance of covariance estimation? We address (1) by deriving TIC: Taylor
Induced Covariance, which captures the randomness of the multivariate
$f_{\theta}(x)$ by incorporating its gradient and curvature around $x$ through
the second order Taylor polynomial. Furthermore, we tackle (2) by introducing
TAC: Task Agnostic Correlations, a metric which leverages conditioning of the
normal distribution to evaluate the covariance. We verify the effectiveness of
TIC through multiple experiments spanning synthetic (univariate, multivariate)
and real-world datasets (UCI Regression, LSP, and MPII Human Pose Estimation).
Our experiments show that TIC outperforms state-of-the-art in accurately
learning the covariance, as quantified through TAC.