Image similarity resources

This page provides resources about image similarity using deep learning, Siamese network, one-shot learning.

There are several ways to compute image similarity with deep learning.

One can either train an end to end deep model which learns similarity between images, or use the Deep model as a feature extractor and then use a standard similarity metric (Dot product, L2 distance etc.)

1)  Using a deep learning model as a feature extractor: This mechanism is very common.

It depends  on the problem domain at hand. For example, if these are natural images with scenes/objects, one can use a pre-trained deep model (say, a GoogleNet model trained on ImageNet or maybe Natural scene images (like, say Places-205) and then extract features from one of the last few layers (either fc6 or fc7). One can extract features and then compute a distance between them. This is also used, for example, in Face recognition or Face verification.

2) A more complicated way of doing this, which also might be more principled or end-to-end, is training a similarity network. These models are called Siamese Networks. Obviously for this, you need to have a lot of data to train these models from scratch.

Siamese network is a neural network that contain two or more identical subnetwork. The purpose of this network is to find the similarity or comparing the relationship between two comparable things. Unlike classification task that uses cross entropy as the loss function, siamese network usually uses contrastive loss or triplet loss.

  • Implementing Siamese Network using Tensorflow with MNIST (GitHub repo)

web page  (its GitHub repo) motivates the author to implement a similar Siamese network using Tensorflow.

  • Using siamese network to do dimensionality reduction and similar image retrieval (GitHub repo)


(mostly) reimplimented this paper (koch et al, Siamese Networks for one-shot learning) in Keras. Trains on the Omniglot dataset.

Also check out the author’s blog post about the paper and one shot learning.


Related Papers:

A Deep Ranking algorithm using Saimese CNN seems to achieve great success compared to most algorithms shown in the paper.