HeAR -- Health Acoustic Representations
Sebastien Baur Zaid Nabulsi Wei-Hung Weng Jake Garrison Louis Blankemeier Sam Fishman Christina Chen Sujay Kakarmath Minyoi Maimbolwa Nsala Sanjase Brian Shuma Yossi Matias Greg S. Corrado Shwetak Patel Shravya Shetty Shruthi Prabhakara Monde Muyoyeta Diego Ardila
Health acoustic sounds such as coughs and breaths are known to contain useful
health signals with significant potential for monitoring health and disease,
yet are underexplored in the medical machine learning community. The existing
deep learning systems for health acoustics are often narrowly trained and
evaluated on a single task, which is limited by data and may hinder
generalization to other tasks. To mitigate these gaps, we develop HeAR, a
scalable self-supervised learning-based deep learning system using masked
autoencoders trained on a large dataset of 313 million two-second long audio
clips. Through linear probes, we establish HeAR as a state-of-the-art health
audio embedding model on a benchmark of 33 health acoustic tasks across 6
datasets. By introducing this work, we hope to enable and accelerate further
health acoustics research.