All Language Models Large and Small
Zhixun Chen Yali Du David Mguni
Many leading language models (LMs) use high-intensity computational resources
both during training and execution. This poses the challenge of lowering
resource costs for deployment and faster execution of decision-making tasks
among others. We introduce a novel plug-and-play LM framework named Language
Optimising Network Distribution (LONDI) framework. LONDI learns to selectively
employ large LMs only where complex decision-making and reasoning are required
while using low-resource LMs everywhere else. LONDI consists of a system of two
(off-)policy networks, an LM, a large LM (LLM), and a reinforcement learning
module that uses switching controls to quickly learn which system states to
call the LLM. We then introduce a variant of LONDI that maintains budget
constraints on LLM calls and hence its resource usage. Theoretically, we prove
LONDI learns the subset of system states to activate the LLM required to solve
the task. We then prove that LONDI converges to optimal solutions while also
preserving budgetary constraints on LLM calls almost surely enabling it to
solve various tasks while significantly lowering computational costs. We test
LONDI's performance in a range of tasks in ScienceWorld and BabyAI-Text and
demonstrate that LONDI can solve tasks only solvable by resource-intensive LLMs
while reducing GPU usage by up to 30%.