Image Similarity using An Ensemble of Context-Sensitive Models
Zukang Liao Min Chen
Image similarity has been extensively studied in computer vision. In recently
years, machine-learned models have shown their ability to encode more semantics
than traditional multivariate metrics. However, in labelling similarity,
assigning a numerical score to a pair of images is less intuitive than
determining if an image A is closer to a reference image R than another image
B. In this work, we present a novel approach for building an image similarity
model based on labelled data in the form of A:R vs B:R. We address the
challenges of sparse sampling in the image space (R, A, B) and biases in the
models trained with context-based data by using an ensemble model. In
particular, we employed two ML techniques to construct such an ensemble model,
namely dimensionality reduction and MLP regressors. Our testing results show
that the ensemble model constructed performs ~5% better than the best
individual context-sensitive models. They also performed better than the model
trained with mixed imagery data as well as existing similarity models, e.g.,
CLIP and DINO. This work demonstrate that context-based labelling and model
training can be effective when an appropriate ensemble approach is used to
alleviate the limitation due to sparse sampling.