{"action":"created","issue":{"url":"https://api.github.com/repos/bemusic/bemuse/issues/85","labels_url":"https://api.github.com/repos/bemusic/bemuse/issues/85/labels{/name}","comments_url":"https://api.github.com/repos/bemusic/bemuse/issues/85/comments","events_url":"https://api.github.com/repos/bemusic/bemuse/issues/85/events","html_url":"https://github.com/bemusic/bemuse/pull/85","id":56481269,"number":85,"title":"Audio offset synchronization and calibration.","user":{"login":"dtinth","id":193136,"avatar_url":"https://avatars.githubusercontent.com/u/193136?v=3","gravatar_id":"","url":"https://api.github.com/users/dtinth","html_url":"https://github.com/dtinth","followers_url":"https://api.github.com/users/dtinth/followers","following_url":"https://api.github.com/users/dtinth/following{/other_user}","gists_url":"https://api.github.com/users/dtinth/gists{/gist_id}","starred_url":"https://api.github.com/users/dtinth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dtinth/subscriptions","organizations_url":"https://api.github.com/users/dtinth/orgs","repos_url":"https://api.github.com/users/dtinth/repos","events_url":"https://api.github.com/users/dtinth/events{/privacy}","received_events_url":"https://api.github.com/users/dtinth/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/bemusic/bemuse/labels/c:in%20progress","name":"c:in progress","color":"2597ec"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-02-04T05:11:50Z","updated_at":"2015-02-04T10:32:01Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/bemusic/bemuse/pulls/85","html_url":"https://github.com/bemusic/bemuse/pull/85","diff_url":"https://github.com/bemusic/bemuse/pull/85.diff","patch_url":"https://github.com/bemusic/bemuse/pull/85.patch"},"body":"Assumptions\r\n-----------------\r\n\r\n- Response time is zero. When the player hears the sound or sees that the note hits the judgement area, he/she pushes the button immediately.\r\n- When the game emits the sound, it takes time ___S___ until the player hears it. (Audio latency)\r\n- When the game renders the display, it takes time ___D___ until the player sees it. (Video latency)\r\n- When the player hits the button, it takes time ___I___ until the computer recognizes it. (Input latency)\r\n\r\n\r\nFindings\r\n----------\r\n\r\n- It is impossible to measure/calibrate the values of ___S___, ___D___, ___I___ separately.\r\n- But it is possible to find these values:\r\n    - ___S___+___I___ (Audio + input latency)\r\n    - ___D___+___I___ (Video + input latency)\r\n    - ___S___-___D___ (Audio-video offset)\r\n\r\n\r\nMethods\r\n----------\r\n\r\n### Calibration\r\n\r\n1. Measure ___A___ = ___S___+___I___\r\n\r\n    This is the time it takes from when the sound is emitted to when the computer recognizes the button press. This means that when computer emits sound at time ___t___, the computer will receive button press at time ___t'___=___t___+___A___. To compensate, when button press is received at time ___t'___, judgement must be performed for time ___t'___-___A___=___t___.\r\n\r\n2. Measure ___B___ = ___S___-___D___\r\n\r\n    This is the audio/video offset. This can be done by letting user adjust the value until audio and video are in sync. With this value, we adjust the display to display notes in time ___t___+___B___.\r\n\r\n     - When computer emits sound at time ___t___, player will hear it at time ___t___+___S___.\r\n     - When computer emits graphics at time ___t___+___B___, player will see it at time ___t___+___D___+___S___-___D___ = ___t___+___S___, thus proving that the user will find audio and video to be in sync.\r\n\r\n### Gameplay\r\n\r\n- `sound(t)`: At time ___t___, emit sound as usual.\r\n- `display(t+B)`: At time ___t___, emit graphic in sync with the sound.\r\n- `judgment(t-A)`: Judge notes ___A___ unit of time behind the sound emission.\r\n\r\n\r\nAuto-Keysounds\r\n---------------------\r\n\r\nThis is a problem if the value of ___A___ is large. If the player hit the note at the correct time, they will hear delayed keysounds.\r\n\r\nThat's why for most keysounded music games, user has to press the button significantly before the sound is emitted. Examples include DJMAX Technika and Tone Sphere.\r\n\r\nTherefore, we have to play keysound for the player, so that the player hears at the correct time. This is called \"Auto-sound\" in Open2Jam.\r\n\r\n### Enabling AutoSound\r\n\r\nAutoSound will only be enabled when value of ___A___ is significantly large. I'd say 16 milliseconds. A warning message should appear in synchronization dialog saying that AutoSound has been enabled to compensate for audio delay.\r\n\r\n### AutoSound Mechanics\r\n\r\n- Each player has a playing state ___p___, defaults to __true__.\r\n    - This state is __true__ when player is actively playing and __false__ when player isn't playing any note.\r\n    - When user doesn't play any note, we don't want keysounds to be automatically emitted, so we stop autosound mechanism until user hits a note again, setting ___p___ back to __true__.\r\n- Each note ___n___ will have a keysound state ___k_(_n_)__, defaults to \"NONE\".\r\n- If player hits the note ___n___ before it is sounded (___k_(_n_)__ is \"NONE\"):\r\n    - Emit the keysound.\r\n    - Set ___k_(_n_)__ to \"EMITTED\"\r\n    - Set ___p___ to __true__\r\n- If it's the time for the note (___t___ = ___t_(_n_)__) and ___k_(_n_)__ is \"NONE\" and the player is playing (___p___ is __true__):\r\n    - Emit the keysound.\r\n    - Set ___k_(_n_)__ to \"EMITTED\"\r\n- If the note is missed and ___k_(_n_)__ is \"EMITTED\":\r\n    - Stop the keysound.\r\n    - Set ___p___ to __false__.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/bemusic/bemuse/issues/comments/72832789","html_url":"https://github.com/bemusic/bemuse/pull/85#issuecomment-72832789","issue_url":"https://api.github.com/repos/bemusic/bemuse/issues/85","id":72832789,"user":{"login":"dtinth","id":193136,"avatar_url":"https://avatars.githubusercontent.com/u/193136?v=3","gravatar_id":"","url":"https://api.github.com/users/dtinth","html_url":"https://github.com/dtinth","followers_url":"https://api.github.com/users/dtinth/followers","following_url":"https://api.github.com/users/dtinth/following{/other_user}","gists_url":"https://api.github.com/users/dtinth/gists{/gist_id}","starred_url":"https://api.github.com/users/dtinth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dtinth/subscriptions","organizations_url":"https://api.github.com/users/dtinth/orgs","repos_url":"https://api.github.com/users/dtinth/repos","events_url":"https://api.github.com/users/dtinth/events{/privacy}","received_events_url":"https://api.github.com/users/dtinth/received_events","type":"User","site_admin":false},"created_at":"2015-02-04T10:32:01Z","updated_at":"2015-02-04T10:32:01Z","body":"Calibrating the value of ___A___\r\n--------------------------------------------\r\n\r\n### Methods\r\n\r\n- Play a song and instruct user to press the button on every beat.\r\n- Record at least 56 samples of ___A___ (time that keypress is registered - time that sound is emitted)\r\n- Analyze the recorded samples."}}