Knowledge Graph Completion Models are Few-shot Learners: An Empirical
  Study of Relation Labeling in E-commerce with LLMs
Jiao Chen Luyi Ma Xiaohan Li Nikhil Thakurdesai Jianpeng Xu Jason H. D. Cho Kaushiki Nag Evren Korpeoglu Sushant Kumar Kannan Achan
Knowledge Graphs (KGs) play a crucial role in enhancing e-commerce system
performance by providing structured information about entities and their
relationships, such as complementary or substitutable relations between
products or product types, which can be utilized in recommender systems.
However, relation labeling in KGs remains a challenging task due to the dynamic
nature of e-commerce domains and the associated cost of human labor. Recently,
breakthroughs in Large Language Models (LLMs) have shown surprising results in
numerous natural language processing tasks. In this paper, we conduct an
empirical study of LLMs for relation labeling in e-commerce KGs, investigating
their powerful learning capabilities in natural language and effectiveness in
predicting relations between product types with limited labeled data. We
evaluate various LLMs, including PaLM and GPT-3.5, on benchmark datasets,
demonstrating their ability to achieve competitive performance compared to
humans on relation labeling tasks using just 1 to 5 labeled examples per
relation. Additionally, we experiment with different prompt engineering
techniques to examine their impact on model performance. Our results show that
LLMs significantly outperform existing KG completion models in relation
labeling for e-commerce KGs and exhibit performance strong enough to replace
human labeling.