PARMESAN: Parameter-Free Memory Search and Transduction for Dense
  Prediction Tasks
Philip Matthias Winter Maria Wimmer David Major Dimitrios Lenis Astrid Berg Theresa Neubauer Gaia Romana De Paolis Johannes Novotny Sophia Ulonska Katja Bühler
In this work we address flexibility in deep learning by means of transductive
reasoning. For adaptation to new tasks or new data, existing methods typically
involve tuning of learnable parameters or even complete re-training from
scratch, rendering such approaches unflexible in practice. We argue that the
notion of separating computation from memory by the means of transduction can
act as a stepping stone for solving these issues. We therefore propose PARMESAN
(parameter-free memory search and transduction), a scalable transduction method
which leverages a memory module for solving dense prediction tasks. At
inference, hidden representations in memory are being searched to find
corresponding examples. In contrast to other methods, PARMESAN learns without
the requirement for any continuous training or fine-tuning of learnable
parameters simply by modifying the memory content. Our method is compatible
with commonly used neural architectures and canonically transfers to 1D, 2D,
and 3D grid-based data. We demonstrate the capabilities of our approach at
complex tasks such as continual and few-shot learning. PARMESAN learns up to
370 times faster than common baselines while being on par in terms of
predictive performance, knowledge retention, and data-efficiency.