{"action":"created","issue":{"url":"https://api.github.com/repos/Russell91/apollo/issues/5","labels_url":"https://api.github.com/repos/Russell91/apollo/issues/5/labels{/name}","comments_url":"https://api.github.com/repos/Russell91/apollo/issues/5/comments","events_url":"https://api.github.com/repos/Russell91/apollo/issues/5/events","html_url":"https://github.com/Russell91/apollo/issues/5","id":96988480,"number":5,"title":"How to handle when softmax outputs do not sum to 1.","user":{"login":"icycandy","id":724257,"avatar_url":"https://avatars.githubusercontent.com/u/724257?v=3","gravatar_id":"","url":"https://api.github.com/users/icycandy","html_url":"https://github.com/icycandy","followers_url":"https://api.github.com/users/icycandy/followers","following_url":"https://api.github.com/users/icycandy/following{/other_user}","gists_url":"https://api.github.com/users/icycandy/gists{/gist_id}","starred_url":"https://api.github.com/users/icycandy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/icycandy/subscriptions","organizations_url":"https://api.github.com/users/icycandy/orgs","repos_url":"https://api.github.com/users/icycandy/repos","events_url":"https://api.github.com/users/icycandy/events{/privacy}","received_events_url":"https://api.github.com/users/icycandy/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-07-24T07:38:12Z","updated_at":"2015-07-30T01:18:37Z","closed_at":"2015-07-26T02:42:33Z","body":"I used the example data in `data/` directory, and run `char_model.py` without any modification.\r\nthe numpy version is 1.9.2\r\n```bash\r\n[candy@f22 apollo]$ md5sum data/language_model/vocab.pkl \r\n450c542ccb75a8fa9b2f86a39ee7db72  data/language_model/vocab.pkl\r\n[candy@f22 apollo]$ md5sum data/char_model/reddit_ml.txt \r\n06f65f172f5d29377955b62a68062abe  data/char_model/reddit_ml.txt\r\n```\r\n\r\n```bash\r\n[candy@f22 apollo]$ python examples/char_model.py \r\n2015-07-24 15:31:56,430 - INFO - Iteration 0: 5.53151462078\r\nTraceback (most recent call last):\r\n  File \"examples/char_model.py\", line 212, in <module>\r\n    eval_performance(net)\r\n  File \"examples/char_model.py\", line 139, in eval_performance\r\n    eval_forward(eval_net)\r\n  File \"examples/char_model.py\", line 165, in eval_forward\r\n    output = softmax_choice(net.tops['softmax'].data)\r\n  File \"examples/char_model.py\", line 145, in softmax_choice\r\n    return np.random.choice(range(len(data.flatten())), p=data.flatten())\r\n  File \"mtrand.pyx\", line 1094, in mtrand.RandomState.choice (numpy/random/mtrand/mtrand.c:10565)\r\nValueError: probabilities do not sum to 1\r\n```"},"comment":{"url":"https://api.github.com/repos/Russell91/apollo/issues/comments/126146415","html_url":"https://github.com/Russell91/apollo/issues/5#issuecomment-126146415","issue_url":"https://api.github.com/repos/Russell91/apollo/issues/5","id":126146415,"user":{"login":"icycandy","id":724257,"avatar_url":"https://avatars.githubusercontent.com/u/724257?v=3","gravatar_id":"","url":"https://api.github.com/users/icycandy","html_url":"https://github.com/icycandy","followers_url":"https://api.github.com/users/icycandy/followers","following_url":"https://api.github.com/users/icycandy/following{/other_user}","gists_url":"https://api.github.com/users/icycandy/gists{/gist_id}","starred_url":"https://api.github.com/users/icycandy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/icycandy/subscriptions","organizations_url":"https://api.github.com/users/icycandy/orgs","repos_url":"https://api.github.com/users/icycandy/repos","events_url":"https://api.github.com/users/icycandy/events{/privacy}","received_events_url":"https://api.github.com/users/icycandy/received_events","type":"User","site_admin":false},"created_at":"2015-07-30T01:18:37Z","updated_at":"2015-07-30T01:18:37Z","body":"I report the bug to numpy, and they seems have fixed it. Before the next release of numpy, currently the workaround is converting `np.dtype` from `float32` to `float 64`\r\n```python\r\ndef softmax_choice(data):\r\n    probs = data.flatten().astype(np.float64)\r\n    probs /= probs.sum()\r\n    return np.random.choice(range(len(probs)), p=probs)\r\n```"}}