Large Language Model Can Continue Evolving From Mistakes
Haokun Zhao Haixia Han Jie Shi Chengyu Du Jiaqing Liang Yanghua Xiao
Large Language Models (LLMs) demonstrate impressive performance in various
downstream tasks. However, they may still generate incorrect responses in
certain scenarios due to the knowledge deficiencies and the flawed pre-training
data. Continual Learning (CL) is a commonly used method to address this issue.
Traditional CL is task-oriented, using novel or factually accurate data to
retrain LLMs from scratch. However, this method requires more task-related
training data and incurs expensive training costs. To address this challenge,
we propose the Continue Evolving from Mistakes (CEM) method, inspired by the
'summarize mistakes' learning skill, to achieve iterative refinement of LLMs.
Specifically, the incorrect responses of LLMs indicate knowledge deficiencies
related to the questions. Therefore, we collect corpora with these knowledge
from multiple data sources and follow it up with iterative supplementary
training for continuous, targeted knowledge updating and supplementation.
Meanwhile, we developed two strategies to construct supplementary training sets
to enhance the LLM's understanding of the corpus and prevent catastrophic
forgetting. We conducted extensive experiments to validate the effectiveness of
this CL method. In the best case, our method resulted in a 17.00\% improvement
in the accuracy of the LLM.