Sparse Function-space Representation of Neural Networks
Aidan Scannell Riccardo Mereu Paul Chang Ella Tamir Joni Pajarinen Arno Solin
Deep neural networks (NNs) are known to lack uncertainty estimates and
struggle to incorporate new data. We present a method that mitigates these
issues by converting NNs from weight space to function space, via a dual
parameterization. Importantly, the dual parameterization enables us to
formulate a sparse representation that captures information from the entire
data set. This offers a compact and principled way of capturing uncertainty and
enables us to incorporate new data without retraining whilst retaining
predictive performance. We provide proof-of-concept demonstrations with the
proposed approach for quantifying uncertainty in supervised learning on UCI
benchmark tasks.