Generation and Detection of Sign Language Deepfakes -- A Linguistic and
  Visual Analysis
Shahzeb Naeem Muhammad Riyyan Khan Usman Tariq Abhinav Dhall Carlos Ivan Colon Hasan Al-Nashash
A question in the realm of deepfakes is slowly emerging pertaining to whether
we can go beyond facial deepfakes and whether it would be beneficial to
society. Therefore, this research presents a positive application of deepfake
technology in upper body generation, while performing sign-language for the
Deaf and Hard of Hearing (DHoH) community. The resulting videos are later
vetted with a sign language expert. This is particularly helpful, given the
intricate nature of sign language, a scarcity of sign language experts, and
potential benefits for health and education. The objectives of this work
encompass constructing a reliable deepfake dataset, evaluating its technical
and visual credibility through computer vision and natural language processing
models, and assessing the plausibility of the generated content. With over 1200
videos, featuring both previously seen and unseen individuals for the
generation model, using the help of a sign language expert, we establish a
deepfake dataset in sign language that can further be utilized to detect fake
videos that may target certain people of determination.