Leveraging Visemes for Better Visual Speech Representation and Lip
  Reading
Javad Peymanfard Vahid Saeedi Mohammad Reza Mohammadi Hossein Zeinali Nasser Mozayani
Lip reading is a challenging task that has many potential applications in
speech recognition, human-computer interaction, and security systems. However,
existing lip reading systems often suffer from low accuracy due to the
limitations of video features. In this paper, we propose a novel approach that
leverages visemes, which are groups of phonetically similar lip shapes, to
extract more discriminative and robust video features for lip reading. We
evaluate our approach on various tasks, including word-level and sentence-level
lip reading, and audiovisual speech recognition using the Arman-AV dataset, a
largescale Persian corpus. Our experimental results show that our viseme based
approach consistently outperforms the state-of-theart methods in all these
tasks. The proposed method reduces the lip-reading word error rate (WER) by
9.1% relative to the best previous method.