Superfiltering: Weak-to-Strong Data Filtering for Fast
  Instruction-Tuning
Ming Li Yong Zhang Shwai He Zhitao Li Hongyu Zhao Jianzong Wang Ning Cheng Tianyi Zhou
Instruction tuning is critical to improve LLMs but usually suffers from
low-quality and redundant data. Data filtering for instruction tuning has
proved important in improving both the efficiency and performance of the tuning
process. But it also leads to extra cost and computation due to the involvement
of LLMs in this process. To reduce the filtering cost, we study Superfiltering:
Can we use a smaller and weaker model to select data for finetuning a larger
and stronger model? Despite the performance gap between weak and strong
language models, we find their highly consistent capability to perceive
instruction difficulty and data selection results. This enables us to use a
much smaller and more efficient model to filter the instruction data used to
train a larger language model. Not only does it largely speed up the data
filtering, but the filtered-data-finetuned LLM achieves even better performance
on standard benchmarks. Extensive experiments validate the efficacy and
efficiency of our approach.