Let's Learn Step by Step: Enhancing In-Context Learning Ability with
  Curriculum Learning
Yinpeng Liu Jiawei Liu Xiang Shi Qikai Cheng Wei Lu
Demonstration ordering, which is an important strategy for in-context
learning (ICL), can significantly affects the performance of large language
models (LLMs). However, most of the current approaches of ordering require
additional knowledge and similarity calculation. We advocate the few-shot
in-context curriculum learning (ICCL), a simple but effective demonstration
ordering method for ICL, which implies gradually increasing the complexity of
prompt demonstrations during the inference process. Then we design three
experiments to discuss the effectiveness of ICCL, the formation mechanism of
LLM's ICCL capability, and the impact of ordering subjects. Experimental
results demonstrate that ICCL, developed during the instruction-tuning stage,
is effective for open-source LLMs. Moreover, LLMs exhibit a weaker capacity
compared to humans in discerning the difficulty levels of demonstrations. We
release our code at https://github.com/61peng/curri_learning.