DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models
Mohammadreza Pourreza Davood Rafiei
Leading models for the text-to-SQL task heavily rely on proprietary Large
Language Models (LLMs), posing concerns over data privacy. Closing the
performance gap between small open-source models and large proprietary models
is crucial to mitigate this reliance. To this end, we introduce a novel
two-stage fine-tuning approach that decomposes the task into two simpler tasks.
Through comprehensive evaluation on two large cross-domain datasets and two
small LLMs, we show that this approach improves execution accuracy by 3 to 7
percent, effectively aligning the performance of open-source models with their
proprietary counterparts.