Fine-tuning Large Language Models with Sequential Instructions
Hanxu Hu Pinzhen Chen Edoardo M. Ponti
Large language models (LLMs) struggle to follow a sequence of instructions in
a single query as they may ignore or misinterpret part of it. This impairs
their performance in complex problems whose solution requires multiple
intermediate steps, such as multilingual (translate then answer) and multimodal
(caption then answer) tasks. We empirically verify this with open-source LLMs
as large as LLaMA-2 70B and Mixtral-8x7B. Targeting the scarcity of sequential
instructions in present-day data, we propose sequential instruction tuning, a
simple yet effective strategy to automatically augment instruction tuning data
and equip LLMs with the ability to execute multiple sequential instructions.
After exploring interleaving instructions in existing datasets, such as Alpaca,
with a wide range of intermediate tasks, we find that sequential
instruction-tuned models consistently outperform the conventional
instruction-tuned baselines in downstream tasks involving reasoning,
multilingual, and multimodal abilities. To shed further light on our technique,
we analyse how adversarial intermediate texts, unseen tasks, prompt
verbalization, number of tasks, and prompt length affect SIT. We hope that this
method will open new research avenues on instruction tuning for complex tasks.