{"action":"created","issue":{"url":"https://api.github.com/repos/amaas/stanford-ctc/issues/3","labels_url":"https://api.github.com/repos/amaas/stanford-ctc/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/amaas/stanford-ctc/issues/3/comments","events_url":"https://api.github.com/repos/amaas/stanford-ctc/issues/3/events","html_url":"https://github.com/amaas/stanford-ctc/issues/3","id":90668539,"number":3,"title":"p(blank symbol) >> p(non-blank symbol) during NN-CTC training","user":{"login":"gmkim90","id":10232337,"avatar_url":"https://avatars.githubusercontent.com/u/10232337?v=3","gravatar_id":"","url":"https://api.github.com/users/gmkim90","html_url":"https://github.com/gmkim90","followers_url":"https://api.github.com/users/gmkim90/followers","following_url":"https://api.github.com/users/gmkim90/following{/other_user}","gists_url":"https://api.github.com/users/gmkim90/gists{/gist_id}","starred_url":"https://api.github.com/users/gmkim90/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gmkim90/subscriptions","organizations_url":"https://api.github.com/users/gmkim90/orgs","repos_url":"https://api.github.com/users/gmkim90/repos","events_url":"https://api.github.com/users/gmkim90/events{/privacy}","received_events_url":"https://api.github.com/users/gmkim90/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-06-24T12:53:32Z","updated_at":"2015-06-25T05:24:34Z","closed_at":null,"body":"Hi all\r\n\r\nI want to discuss some issue regarding training DNN/CNN-CTC for speech recognition. (Wall Street Journal Corpus). I modeled output unit as characters.\r\n\r\nI observed that CTC objective function was increasing and finally converged during training.\r\n![image](https://cloud.githubusercontent.com/assets/10232337/8330191/8a6a7da4-1aba-11e5-92ce-752303d35503.png)\r\n\r\n\r\nBut I also observed that final NN outputs have clear tendency : p(blank symbol) >> p(non-blank symbol) for all speech time frame as following figure\r\n\r\n![image](https://cloud.githubusercontent.com/assets/10232337/8330115/1debd722-1aba-11e5-9772-81f099dd4fda.png)\r\n\r\nIn Alex Graves' paper, trained RNN should have high p(non-blank) at some point like following figure\r\n![image](https://cloud.githubusercontent.com/assets/10232337/8330169/6d3d2f92-1aba-11e5-8f8e-f70c2c8b9419.png)\r\n\r\n\r\nDo you have same situation when you train NN-CTC for sequence labeling problem?  I am suspecting that the reason is I use MLP/CNN instead of RNN, but I can't clearly explain why this can be a reason.   \r\nAny idea about this result?\r\n\r\nThank you for reading my question.\r\n\r\n\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/amaas/stanford-ctc/issues/comments/115108797","html_url":"https://github.com/amaas/stanford-ctc/issues/3#issuecomment-115108797","issue_url":"https://api.github.com/repos/amaas/stanford-ctc/issues/3","id":115108797,"user":{"login":"zxie","id":191424,"avatar_url":"https://avatars.githubusercontent.com/u/191424?v=3","gravatar_id":"","url":"https://api.github.com/users/zxie","html_url":"https://github.com/zxie","followers_url":"https://api.github.com/users/zxie/followers","following_url":"https://api.github.com/users/zxie/following{/other_user}","gists_url":"https://api.github.com/users/zxie/gists{/gist_id}","starred_url":"https://api.github.com/users/zxie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zxie/subscriptions","organizations_url":"https://api.github.com/users/zxie/orgs","repos_url":"https://api.github.com/users/zxie/repos","events_url":"https://api.github.com/users/zxie/events{/privacy}","received_events_url":"https://api.github.com/users/zxie/received_events","type":"User","site_admin":false},"created_at":"2015-06-25T05:24:34Z","updated_at":"2015-06-25T05:24:34Z","body":"Your MLP gives framewise predictions, correct? Could you detail how your cost is computed w.r.t. the desired character sequence? Are you just using (T - CW / 2) CNN-MLPs, where T denotes the number of input frames?"}}