Contextualising Implicit Representations for Semantic Tasks
Theo W. Costain Kejie Li Victor A. Prisacariu
Prior works have demonstrated that implicit representations trained only for
reconstruction tasks typically generate encodings that are not useful for
semantic tasks. In this work, we propose a method that contextualises the
encodings of implicit representations, enabling their use in downstream tasks
(e.g. semantic segmentation), without requiring access to the original training
data or encoding network. Using an implicit representation trained for a
reconstruction task alone, our contextualising module takes an encoding trained
for reconstruction only and reveals meaningful semantic information that is
hidden in the encodings, without compromising the reconstruction performance.
With our proposed module, it becomes possible to pre-train implicit
representations on larger datasets, improving their reconstruction performance
compared to training on only a smaller labelled dataset, whilst maintaining
their segmentation performance on the labelled dataset. Importantly, our method
allows for future foundation implicit representation models to be fine-tuned on
unseen tasks, regardless of encoder or dataset availability.