I$^2$KD-SLU: An Intra-Inter Knowledge Distillation Framework for
  Zero-Shot Cross-Lingual Spoken Language Understanding
Tianjun Mao Chenghong Zhang
Spoken language understanding (SLU) typically includes two subtasks: intent
detection and slot filling. Currently, it has achieved great success in
high-resource languages, but it still remains challenging in low-resource
languages due to the scarcity of labeled training data. Hence, there is a
growing interest in zero-shot cross-lingual SLU. Despite of the success of
existing zero-shot cross-lingual SLU models, most of them neglect to achieve
the mutual guidance between intent and slots. To address this issue, we propose
an Intra-Inter Knowledge Distillation framework for zero-shot cross-lingual
Spoken Language Understanding (I$^2$KD-SLU) to model the mutual guidance.
Specifically, we not only apply intra-knowledge distillation between intent
predictions or slot predictions of the same utterance in different languages,
but also apply inter-knowledge distillation between intent predictions and slot
predictions of the same utterance. Our experimental results demonstrate that
our proposed framework significantly improves the performance compared with the
strong baselines and achieves the new state-of-the-art performance on the
MultiATIS++ dataset, obtaining a significant improvement over the previous best
model in overall accuracy.