Rethinking Data Selection for Supervised Fine-Tuning
Ming Shen
Although supervised finetuning (SFT) has emerged as an essential technique to
align large language models with humans, it is considered superficial, with
style learning being its nature. At the same time, recent works indicate the
importance of data selection for SFT, showing that finetuning with high-quality
and diverse subsets of the original dataset leads to superior downstream
performance. In this work, we rethink the intuition behind data selection for
SFT. Considering SFT is superficial, we propose that essential demonstrations
for SFT should focus on reflecting human-like interactions instead of data
quality or diversity. However, it is not straightforward to directly assess to
what extent a demonstration reflects human styles. Towards an initial attempt
in this direction, we find selecting instances with long responses is
surprisingly more effective for SFT than utilizing full datasets or instances
selected based on quality and diversity. We hypothesize that such a simple
heuristic implicitly mimics a crucial aspect of human-style conversation:
detailed responses are usually more helpful.