SparseVSR: Lightweight and Noise Robust Visual Speech Recognition
Adriana Fernandez-Lopez Honglie Chen Pingchuan Ma Alexandros Haliassos Stavros Petridis Maja Pantic
Recent advances in deep neural networks have achieved unprecedented success
in visual speech recognition. However, there remains substantial disparity
between current methods and their deployment in resource-constrained devices.
In this work, we explore different magnitude-based pruning techniques to
generate a lightweight model that achieves higher performance than its dense
model equivalent, especially under the presence of visual noise. Our sparse
models achieve state-of-the-art results at 10% sparsity on the LRS3 dataset and
outperform the dense equivalent up to 70% sparsity. We evaluate our 50% sparse
model on 7 different visual noise types and achieve an overall absolute
improvement of more than 2% WER compared to the dense equivalent. Our results
confirm that sparse networks are more resistant to noise than dense networks.