Unsupervised Speaker Diarization in Distributed IoT Networks Using
  Federated Learning
Amit Kumar Bhuyan Hrishikesh Dutta Subir Biswas
This paper presents a computationally efficient and distributed speaker
diarization framework for networked IoT-style audio devices. The work proposes
a Federated Learning model which can identify the participants in a
conversation without the requirement of a large audio database for training. An
unsupervised online update mechanism is proposed for the Federated Learning
model which depends on cosine similarity of speaker embeddings. Moreover, the
proposed diarization system solves the problem of speaker change detection via.
unsupervised segmentation techniques using Hotelling's t-squared Statistic and
Bayesian Information Criterion. In this new approach, speaker change detection
is biased around detected quasi-silences, which reduces the severity of the
trade-off between the missed detection and false detection rates. Additionally,
the computational overhead due to frame-by-frame identification of speakers is
reduced via. unsupervised clustering of speech segments. The results
demonstrate the effectiveness of the proposed training method in the presence
of non-IID speech data. It also shows a considerable improvement in the
reduction of false and missed detection at the segmentation stage, while
reducing the computational overhead. Improved accuracy and reduced
computational cost makes the mechanism suitable for real-time speaker
diarization across a distributed IoT audio network.