CANDLE: Iterative Conceptualization and Instantiation Distillation from
  Large Language Models for Commonsense Reasoning
Weiqi Wang Tianqing Fang Chunyang Li Haochen Shi Wenxuan Ding Baixuan Xu Zhaowei Wang Jiaxin Bai Xin Liu Jiayang Cheng Chunkit Chan Yangqiu Song
The sequential process of conceptualization and instantiation is essential to
generalizable commonsense reasoning as it allows the application of existing
knowledge to unfamiliar scenarios. However, existing works tend to undervalue
the step of instantiation and heavily rely on pre-built concept taxonomies and
human annotations to collect both types of knowledge, resulting in a lack of
instantiated knowledge to complete reasoning, high cost, and limited
scalability. To tackle these challenges, we introduce CANDLE, a distillation
framework that iteratively performs contextualized conceptualization and
instantiation over commonsense knowledge bases by instructing large language
models to generate both types of knowledge with critic filtering. By applying
CANDLE to ATOMIC, we construct a comprehensive knowledge base comprising six
million conceptualizations and instantiated commonsense knowledge triples. Both
types of knowledge are firmly rooted in the original ATOMIC dataset, and
intrinsic evaluations demonstrate their exceptional quality and diversity.
Empirical results indicate that distilling CANDLE on student models provides
benefits across four downstream tasks. Our code, data, and models are publicly
available at https://github.com/HKUST-KnowComp/CANDLE.