{"action":"created","issue":{"url":"https://api.github.com/repos/mila-udem/blocks/issues/654","labels_url":"https://api.github.com/repos/mila-udem/blocks/issues/654/labels{/name}","comments_url":"https://api.github.com/repos/mila-udem/blocks/issues/654/comments","events_url":"https://api.github.com/repos/mila-udem/blocks/issues/654/events","html_url":"https://github.com/mila-udem/blocks/issues/654","id":78932390,"number":654,"title":"Question about `Softmax.categorical_crossentopy`","user":{"login":"rizar","id":654434,"avatar_url":"https://avatars.githubusercontent.com/u/654434?v=3","gravatar_id":"","url":"https://api.github.com/users/rizar","html_url":"https://github.com/rizar","followers_url":"https://api.github.com/users/rizar/followers","following_url":"https://api.github.com/users/rizar/following{/other_user}","gists_url":"https://api.github.com/users/rizar/gists{/gist_id}","starred_url":"https://api.github.com/users/rizar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rizar/subscriptions","organizations_url":"https://api.github.com/users/rizar/orgs","repos_url":"https://api.github.com/users/rizar/repos","events_url":"https://api.github.com/users/rizar/events{/privacy}","received_events_url":"https://api.github.com/users/rizar/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/mila-udem/blocks/labels/question","name":"question","color":"cc317c"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-05-21T10:27:53Z","updated_at":"2015-07-14T11:19:48Z","closed_at":"2015-05-22T17:03:58Z","body":"I guess I do not get something simple, but I will still ask: why are we not using `theano.nnet.softmax` [there](https://github.com/mila-udem/blocks/blob/master/blocks/bricks/__init__.py#L503)?"},"comment":{"url":"https://api.github.com/repos/mila-udem/blocks/issues/comments/121205752","html_url":"https://github.com/mila-udem/blocks/issues/654#issuecomment-121205752","issue_url":"https://api.github.com/repos/mila-udem/blocks/issues/654","id":121205752,"user":{"login":"mjwillson","id":4502,"avatar_url":"https://avatars.githubusercontent.com/u/4502?v=3","gravatar_id":"","url":"https://api.github.com/users/mjwillson","html_url":"https://github.com/mjwillson","followers_url":"https://api.github.com/users/mjwillson/followers","following_url":"https://api.github.com/users/mjwillson/following{/other_user}","gists_url":"https://api.github.com/users/mjwillson/gists{/gist_id}","starred_url":"https://api.github.com/users/mjwillson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mjwillson/subscriptions","organizations_url":"https://api.github.com/users/mjwillson/orgs","repos_url":"https://api.github.com/users/mjwillson/repos","events_url":"https://api.github.com/users/mjwillson/events{/privacy}","received_events_url":"https://api.github.com/users/mjwillson/received_events","type":"User","site_admin":false},"created_at":"2015-07-14T11:19:48Z","updated_at":"2015-07-14T11:19:48Z","body":"@lamblin I noticed in theano itself there is `theano.tensor.nnet.categorical_crossentropy`.\r\n\r\nFor the one-hot case of this uses CrossentropyCategorical1Hot, which seems to have a grad implemented, and its docs state:\r\n\r\n```\r\n    :note: In the case that the coding distribution is the output of a\r\n           softmax, an application of this Op will probably be optimized\r\n           away in favour of one with a C implementation.\r\n```\r\n\r\nWhile I don't like the sound of \"probably\", is there a reason not to use this on top of `theano.tensor.nnet.softmax` in `Softmax.categorical_crossentopy` ? Is there an easy way I can find out what optimisations it's made under the hood?\r\n\r\nIt'd be good to clear this stuff up anyway, either here or in theano. I'm not enough of an expert to debug it authoritatively, but for me using blocks' Softmax.categorical_crossentopy seems to cause theano to do something suspiciously slow / inefficient (turning on profiling, around 2/3 of the time is spent in Softmax and Elemwise exp, which seems way too much...)"}}