Leveraging Habitat Information for Fine-grained Bird Identification
Tin Nguyen Anh Nguyen
Traditional bird classifiers mostly rely on the visual characteristics of
birds. Some prior works even train classifiers to be invariant to the
background, completely discarding the living environment of birds. Instead, we
are the first to explore integrating habitat information, one of the four major
cues for identifying birds by ornithologists, into modern bird classifiers. We
focus on two leading model types: (1) CNNs and ViTs trained on the downstream
bird datasets; and (2) original, multi-modal CLIP. Training CNNs and ViTs with
habitat-augmented data results in an improvement of up to +0.83 and +0.23
points on NABirds and CUB-200, respectively. Similarly, adding habitat
descriptors to the prompts for CLIP yields a substantial accuracy boost of up
to +0.99 and +1.1 points on NABirds and CUB-200, respectively. We find
consistent accuracy improvement after integrating habitat features into the
image augmentation process and into the textual descriptors of vision-language
CLIP classifiers. Code is available at:
https://anonymous.4open.science/r/reasoning-8B7E/.