ChatGPT-Powered Hierarchical Comparisons for Image Classification
Zhiyuan Ren Yiyang Su Xiaoming Liu
The zero-shot open-vocabulary challenge in image classification is tackled by
pretrained vision-language models like CLIP, which benefit from incorporating
class-specific knowledge from large language models (LLMs) like ChatGPT.
However, biases in CLIP lead to similar descriptions for distinct but related
classes, prompting our novel image classification framework via hierarchical
comparisons: using LLMs to recursively group classes into hierarchies and
classifying images by comparing image-text embeddings at each hierarchy level,
resulting in an intuitive, effective, and explainable approach.