Implicit Self-supervised Language Representation for Spoken Language
  Diarization
Jagabandhu Mishra S. R. Mahadeva Prasanna
In a code-switched (CS) scenario, the use of spoken language diarization (LD)
as a pre-possessing system is essential. Further, the use of implicit
frameworks is preferable over the explicit framework, as it can be easily
adapted to deal with low/zero resource languages. Inspired by speaker
diarization (SD) literature, three frameworks based on (1) fixed segmentation,
(2) change point-based segmentation and (3) E2E are proposed to perform LD. The
initial exploration with synthetic TTSF-LD dataset shows, using x-vector as
implicit language representation with appropriate analysis window length ($N$)
can able to achieve at per performance with explicit LD. The best implicit LD
performance of $6.38$ in terms of Jaccard error rate (JER) is achieved by using
the E2E framework. However, considering the E2E framework the performance of
implicit LD degrades to $60.4$ while using with practical Microsoft CS (MSCS)
dataset. The difference in performance is mostly due to the distributional
difference between the monolingual segment duration of secondary language in
the MSCS and TTSF-LD datasets. Moreover, to avoid segment smoothing, the
smaller duration of the monolingual segment suggests the use of a small value
of $N$. At the same time with small $N$, the x-vector representation is unable
to capture the required language discrimination due to the acoustic similarity,
as the same speaker is speaking both languages. Therefore, to resolve the issue
a self-supervised implicit language representation is proposed in this study.
In comparison with the x-vector representation, the proposed representation
provides a relative improvement of $63.9\%$ and achieved a JER of $21.8$ using
the E2E framework.