EchoVest: Real-Time Sound Classification and Depth Perception Expressed
  through Transcutaneous Electrical Nerve Stimulation
Jesse Choe Siddhant Sood Ryan Park
Over 1.5 billion people worldwide live with hearing impairment. Despite
various technologies that have been created for individuals with such
disabilities, most of these technologies are either extremely expensive or
inaccessible for everyday use in low-medium income countries. In order to
combat this issue, we have developed a new assistive device, EchoVest, for
blind/deaf people to intuitively become more aware of their environment.
EchoVest transmits vibrations to the user's body by utilizing transcutaneous
electric nerve stimulation (TENS) based on the source of the sounds. EchoVest
also provides various features, including sound localization, sound
classification, noise reduction, and depth perception. We aimed to outperform
CNN-based machine-learning models, the most commonly used machine learning
model for classification tasks, in accuracy and computational costs. To do so,
we developed and employed a novel audio pipeline that adapts the Audio
Spectrogram Transformer (AST) model, an attention-based model, for our sound
classification purposes, and Fast Fourier Transforms for noise reduction. The
application of Otsu's Method helped us find the optimal thresholds for
background noise sound filtering and gave us much greater accuracy. In order to
calculate direction and depth accurately, we applied Complex Time Difference of
Arrival algorithms and SOTA localization. Our last improvement was to use blind
source separation to make our algorithms applicable to multiple microphone
inputs. The final algorithm achieved state-of-the-art results on numerous
checkpoints, including a 95.7\% accuracy on the ESC-50 dataset for
environmental sound classification.