Diverse Neural Audio Embeddings -- Bringing Features back !
Prateek Verma
With the advent of modern AI architectures, a shift has happened towards
end-to-end architectures. This pivot has led to neural architectures being
trained without domain-specific biases/knowledge, optimized according to the
task. We in this paper, learn audio embeddings via diverse feature
representations, in this case, domain-specific. For the case of audio
classification over hundreds of categories of sound, we learn robust separate
embeddings for diverse audio properties such as pitch, timbre, and neural
representation, along with also learning it via an end-to-end architecture. We
observe handcrafted embeddings, e.g., pitch and timbre-based, although on their
own, are not able to beat a fully end-to-end representation, yet adding these
together with end-to-end embedding helps us, significantly improve performance.
This work would pave the way to bring some domain expertise with end-to-end
models to learn robust, diverse representations, surpassing the performance of
just training end-to-end models.