{"action":"created","issue":{"url":"https://api.github.com/repos/WebAudio/web-audio-api/issues/468","labels_url":"https://api.github.com/repos/WebAudio/web-audio-api/issues/468/labels{/name}","comments_url":"https://api.github.com/repos/WebAudio/web-audio-api/issues/468/comments","events_url":"https://api.github.com/repos/WebAudio/web-audio-api/issues/468/events","html_url":"https://github.com/WebAudio/web-audio-api/issues/468","id":55779414,"number":468,"title":"AudioFrequencyWorkerNode for working in the Frequency Domain","user":{"login":"hughrawlinson","id":829836,"avatar_url":"https://avatars.githubusercontent.com/u/829836?v=3","gravatar_id":"","url":"https://api.github.com/users/hughrawlinson","html_url":"https://github.com/hughrawlinson","followers_url":"https://api.github.com/users/hughrawlinson/followers","following_url":"https://api.github.com/users/hughrawlinson/following{/other_user}","gists_url":"https://api.github.com/users/hughrawlinson/gists{/gist_id}","starred_url":"https://api.github.com/users/hughrawlinson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hughrawlinson/subscriptions","organizations_url":"https://api.github.com/users/hughrawlinson/orgs","repos_url":"https://api.github.com/users/hughrawlinson/repos","events_url":"https://api.github.com/users/hughrawlinson/events{/privacy}","received_events_url":"https://api.github.com/users/hughrawlinson/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":30,"created_at":"2015-01-28T16:31:22Z","updated_at":"2015-02-24T11:50:27Z","closed_at":null,"body":"At the moment, ScriptProcessorNodes and AudioWorkers are operating on the time domain buffer data. At the Web Audio Conference, it seems like there's demand for frequency domain data inside a callback that's going to get called for every audio frame.\r\n\r\nWe're thus proposing an new node called an AudioFrequencyWorkerNode, which gives the developer the option to obtain audio data, perform processing, and output in the frequency domain. This involves passing an options object to the createAudioFrequencyWorker method, specifying the input and output types.\r\n\r\n### Defaults\r\nThe AudioFrequencyWorkerNode should allow access to time domain and frequency domain data concurrently. If no options object is passed to createAudioFrequencyWorker, the input type would default to the amplitude/phase pair, as would the output. However, the options object would allow the user to choose between amplitude/phase, real/imaginary, and time domain data. The dataOut would default to the same as the dataIn, but could be set to a different data type, in case the user wants to read real/imaginary pairs in, and write out to the time domain for example.\r\n\r\n### Proposed processing structure of the AudioFrequencyWorkerNode\r\n\r\nINPUT (time-domain)\r\n↓\r\nwindowing\r\n↓\r\nFFT\r\n↓\r\n~ ~ ~ ~ ~ ~ ~ ~ ~\r\ndataIn\r\n↓\r\nonaudioprocess\r\n↓\r\ndataOut\r\n~ ~ ~ ~ ~ ~ ~ ~ ~\r\n↓\r\nmirror\r\n↓\r\ncomplete data\r\n↓\r\nIFFT\r\n↓\r\nwindowing\r\n↓\r\nOUTPUT (time-domain)\r\n\r\n\r\n### Example code:\r\n\r\n_main JS_\r\n\r\n```javascript\r\nvar aw = createAudioFrequencyWorker(\"worker.js\",{\r\n        dataIn:[\r\n            \"amplitude\",\r\n            \"phase\",\r\n            \"real\",\r\n            \"imaginary\",\r\n            \"signal\"\r\n        ],\r\n        dataOut:\"complex\" || \"amplitude\" || \"phase\" || \"signal\",\r\n        bufferSize:[power of two],\r\n        hopSize:N,\r\n        windowingType,\r\n        zeroPadding:N\r\n    }\r\n// Signal (time domain) would be the default I/O for the AudioFrequencyWorker.\r\n// indicating another datatype would actuate an fft and/or ifft around the user code.\r\n```\r\n\r\n_AudioWorker JS_\r\n\r\n```javascript\r\n// callback to AudioFrequencyWorkerNode\r\nonaudioprocess = function (e) {\r\n  // e.amplitude[0][channel];\r\n  // e.phase[0][channel];\r\n  // e.real[0][channel];\r\n  // e.imaginary[0][channel];\r\n  // e.signal[0][channel]; // replacing the 'input', time domain.\r\n  \r\n  // edit the arrays in place, rather than having a separate output array to copy into\r\n  \r\n  for (var channel = 0; channel<e.amplitude.length;channel++){\r\n    for(var i = 0; i < e.amplitude.length;i++){\r\n        e.amplitude[i][channel] = e.amplitude[i][channel]>0.5 ? e.amplitude[i][channel] : 0;\r\n    }\r\n  }\r\n};\r\n```\r\n\r\n### Use Cases\r\n* Feature Extraction,\r\n* Phase Vocoding,\r\n* Pitch Shifting,\r\n* Pitch Track,\r\n* Time stretching,\r\n* Frequency domain gating effects\r\n\r\nJesse Allison, Hugh Rawlinson, Jakub Fiala, Nevo Segal\r\n@jesseallison, @hughrawlinson, @jakubfiala, @nevosegal\r\n\r\n### Related\r\n#248 \r\n#262 \r\n"},"comment":{"url":"https://api.github.com/repos/WebAudio/web-audio-api/issues/comments/75743118","html_url":"https://github.com/WebAudio/web-audio-api/issues/468#issuecomment-75743118","issue_url":"https://api.github.com/repos/WebAudio/web-audio-api/issues/468","id":75743118,"user":{"login":"jakubfiala","id":5719805,"avatar_url":"https://avatars.githubusercontent.com/u/5719805?v=3","gravatar_id":"","url":"https://api.github.com/users/jakubfiala","html_url":"https://github.com/jakubfiala","followers_url":"https://api.github.com/users/jakubfiala/followers","following_url":"https://api.github.com/users/jakubfiala/following{/other_user}","gists_url":"https://api.github.com/users/jakubfiala/gists{/gist_id}","starred_url":"https://api.github.com/users/jakubfiala/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jakubfiala/subscriptions","organizations_url":"https://api.github.com/users/jakubfiala/orgs","repos_url":"https://api.github.com/users/jakubfiala/repos","events_url":"https://api.github.com/users/jakubfiala/events{/privacy}","received_events_url":"https://api.github.com/users/jakubfiala/received_events","type":"User","site_admin":false},"created_at":"2015-02-24T11:50:27Z","updated_at":"2015-02-24T11:50:27Z","body":" @notthetup awesome! As far as I understand, the `fft-asm.js` option isn't working yet, right?"}}