Aikyam: A Video Conferencing Utility for Deaf and Dumb
Kshitij Deshpande Varad Mashalkar Kaustubh Mhaisekar Amaan Naikwadi Archana Ghotkar
With the advent of the pandemic, the use of video conferencing platforms as a
means of communication has greatly increased and with it, so have the remote
opportunities. The deaf and dumb have traditionally faced several issues in
communication, but now the effect is felt more severely. This paper proposes an
all-encompassing video conferencing utility that can be used with existing
video conferencing platforms to address these issues. Appropriate semantically
correct sentences are generated from the signer's gestures which would be
interpreted by the system. Along with an audio to emit this sentence, the
user's feed is also used to annotate the sentence. This can be viewed by all
participants, thus aiding smooth communication with all parties involved. This
utility utilizes a simple LSTM model for classification of gestures. The
sentences are constructed by a t5 based model. In order to achieve the required
data flow, a virtual camera is used.