Streaming multi-track audio using DASH
<p>We are creating a web browser based music streaming service which streams files containing separate, synchronzied audio tracks. The files are basically mp4s, but with some additional boxes.</p>

<p>Streaming will be done following the MPEG-DASH standard, and hopefully without reinventing the wheel (for example, there's <a href="https://github.com/Dash-Industry-Forum/dash.js" rel="noreferrer">dash.js</a>).</p>

<hr>

<p><em>Update 160525 - Adds some more background:</em></p>

<p>MPEG-DASH is a requirement for the project, both for its features (adaptiveness, easy server setups etc.) and because the project I'm working on only uses MPEG standards.</p>

<p>The streaming itself is on-demand playback of static files, i.e. no live streaming. The playback will have two modes; a static mode which just plays the tracks, and an interactive mode where the user will be able to change volume, panning etc. For now, static playback is fine.</p>

<hr>

<p>Now, streaming synced multi-track audio doesn't seem to be the most common thing out there, and I've run into a couple of problems:</p>

<h3>1. Following the MPEG-DASH standard</h3>

<p>MPEG-DASH seems to be to be highly focused on video streaming.</p>

<p><em>Are there any documents or guidelines regarding audio-only streaming that I'm not able to find?</em></p>

<h3>2. HTML5 elements vs Web Audio API</h3>

<p>AFAIK, the <code>&lt;video&gt;</code> and <code>&lt;audio&gt;</code> elements only supports one audio track at a time. They provide lots of magic via <code>MediaSource.addSourceBuffer()</code>, where on-the-fly decoding of incoming buffers is taken care of.</p>

<p>Two possible ways of getting around this is:</p>

<ol>
<li>Use multiple <code>&lt;audio&gt;</code> elements and sync them manually</li>
<li>Use the Web Audio API to (a) combine the buffers into a single signal sent to the audio context's destination, or (b) create <code>AudioNode</code>s for each track and sync them manually. This means we have to take care of decoding</li>
</ol>

<p><em>Are there any other or better ways to stream multiple, synchronized audio tracks?</em></p>

<h3>3. Using existing libraries</h3>

<p>The dash.js reference player follows the DASH-AVC/264 guidelines, which to my understanding addresses video streaming and restricts the number of simultaneously playback audio tracks to one.</p>

<p>It also uses the HTML5 <code>&lt;video&gt;</code> element, which leads to the issues in 2. above.</p>

<p>It does, however, contain <strong>huge</strong> amounts of DASH related features, such as adaptive streaming, MPD manifest parsing etc etc.</p>

<p><em>Is dash.js a good suit for multi-track music streaming, or will it end up being a hack?</em></p>

<hr>

<p>I am more than happy to answer questions or be corrected if I have misunderstood anything!</p>