InstrumentGen: Generating Sample-Based Musical Instruments From Text
Shahan Nercessian Johannes Imort
We introduce the text-to-instrument task, which aims at generating
sample-based musical instruments based on textual prompts. Accordingly, we
propose InstrumentGen, a model that extends a text-prompted generative audio
framework to condition on instrument family, source type, pitch (across an
88-key spectrum), velocity, and a joint text/audio embedding. Furthermore, we
present a differentiable loss function to evaluate the intra-instrument timbral
consistency of sample-based instruments. Our results establish a foundational
text-to-instrument baseline, extending research in the domain of automatic
sample-based instrument generation.