LangBridge: Multilingual Reasoning Without Multilingual Supervision
Dongkeun Yoon Joel Jang Sungdong Kim Seungone Kim Sheikh Shafayat Minjoon Seo
We introduce LangBridge, a zero-shot approach to adapt language models for
multilingual reasoning tasks without multilingual supervision. LangBridge
operates by bridging two models, each specialized in different aspects: (1) one
specialized in understanding multiple languages (e.g., mT5 encoder) and (2) one
specialized in reasoning (e.g., Orca 2). LangBridge connects the two models by
introducing minimal trainable parameters between them. Despite utilizing only
English data for training, LangBridge considerably enhances the performance of
language models on low-resource languages across mathematical reasoning,
coding, and logical reasoning. Our analysis suggests that the efficacy of
LangBridge stems from the language-agnostic characteristics of multilingual
representations. We publicly release our code and models.