Language Semantic Graph Guided Data-Efficient Learning
Wenxuan Ma Shuang Li Lincan Cai Jingxuan Kang
Developing generalizable models that can effectively learn from limited data
and with minimal reliance on human supervision is a significant objective
within the machine learning community, particularly in the era of deep neural
networks. Therefore, to achieve data-efficient learning, researchers typically
explore approaches that can leverage more related or unlabeled data without
necessitating additional manual labeling efforts, such as Semi-Supervised
Learning (SSL), Transfer Learning (TL), and Data Augmentation (DA). SSL
leverages unlabeled data in the training process, while TL enables the transfer
of expertise from related data distributions. DA broadens the dataset by
synthesizing new data from existing examples. However, the significance of
additional knowledge contained within labels has been largely overlooked in
research. In this paper, we propose a novel perspective on data efficiency that
involves exploiting the semantic information contained in the labels of the
available data. Specifically, we introduce a Language Semantic Graph (LSG)
which is constructed from labels manifest as natural language descriptions.
Upon this graph, an auxiliary graph neural network is trained to extract
high-level semantic relations and then used to guide the training of the
primary model, enabling more adequate utilization of label knowledge. Across
image, video, and audio modalities, we utilize the LSG method in both TL and
SSL scenarios and illustrate its versatility in significantly enhancing
performance compared to other data-efficient learning approaches. Additionally,
our in-depth analysis shows that the LSG method also expedites the training
process.