Unsupervised Object Localization with Representer Point Selection
Yeonghwan Song Seokwoo Jang Dina Katabi Jeany Son
We propose a novel unsupervised object localization method that allows us to
explain the predictions of the model by utilizing self-supervised pre-trained
models without additional finetuning. Existing unsupervised and self-supervised
object localization methods often utilize class-agnostic activation maps or
self-similarity maps of a pre-trained model. Although these maps can offer
valuable information for localization, their limited ability to explain how the
model makes predictions remains challenging. In this paper, we propose a simple
yet effective unsupervised object localization method based on representer
point selection, where the predictions of the model can be represented as a
linear combination of representer values of training points. By selecting
representer points, which are the most important examples for the model
predictions, our model can provide insights into how the model predicts the
foreground object by providing relevant examples as well as their importance.
Our method outperforms the state-of-the-art unsupervised and self-supervised
object localization methods on various datasets with significant margins and
even outperforms recent weakly supervised and few-shot methods.