{"action":"created","issue":{"url":"https://api.github.com/repos/torch/nn/issues/413","labels_url":"https://api.github.com/repos/torch/nn/issues/413/labels{/name}","comments_url":"https://api.github.com/repos/torch/nn/issues/413/comments","events_url":"https://api.github.com/repos/torch/nn/issues/413/events","html_url":"https://github.com/torch/nn/pull/413","id":109647653,"number":413,"title":"Adds SpatialCrossLRN","user":{"login":"fmassa","id":9110200,"avatar_url":"https://avatars.githubusercontent.com/u/9110200?v=3","gravatar_id":"","url":"https://api.github.com/users/fmassa","html_url":"https://github.com/fmassa","followers_url":"https://api.github.com/users/fmassa/followers","following_url":"https://api.github.com/users/fmassa/following{/other_user}","gists_url":"https://api.github.com/users/fmassa/gists{/gist_id}","starred_url":"https://api.github.com/users/fmassa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fmassa/subscriptions","organizations_url":"https://api.github.com/users/fmassa/orgs","repos_url":"https://api.github.com/users/fmassa/repos","events_url":"https://api.github.com/users/fmassa/events{/privacy}","received_events_url":"https://api.github.com/users/fmassa/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-10-03T21:47:48Z","updated_at":"2015-10-19T04:13:50Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/torch/nn/pulls/413","html_url":"https://github.com/torch/nn/pull/413","diff_url":"https://github.com/torch/nn/pull/413.diff","patch_url":"https://github.com/torch/nn/pull/413.patch"},"body":"Implements [Local Response Normalization between different channels](https://code.google.com/p/cuda-convnet2/wiki/LayerParams#Local_response_normalization_layer_%28across_maps%29).\r\nCould eventually be adapted to use the CUDA implementation in [inn](https://github.com/szagoruyko/imagine-nn/blob/master/SpatialCrossResponseNormalization.cu).\r\nI named it as in `cudnn`, because it's shorter than `inn`s name."},"comment":{"url":"https://api.github.com/repos/torch/nn/issues/comments/149094441","html_url":"https://github.com/torch/nn/pull/413#issuecomment-149094441","issue_url":"https://api.github.com/repos/torch/nn/issues/413","id":149094441,"user":{"login":"fmassa","id":9110200,"avatar_url":"https://avatars.githubusercontent.com/u/9110200?v=3","gravatar_id":"","url":"https://api.github.com/users/fmassa","html_url":"https://github.com/fmassa","followers_url":"https://api.github.com/users/fmassa/followers","following_url":"https://api.github.com/users/fmassa/following{/other_user}","gists_url":"https://api.github.com/users/fmassa/gists{/gist_id}","starred_url":"https://api.github.com/users/fmassa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fmassa/subscriptions","organizations_url":"https://api.github.com/users/fmassa/orgs","repos_url":"https://api.github.com/users/fmassa/repos","events_url":"https://api.github.com/users/fmassa/events{/privacy}","received_events_url":"https://api.github.com/users/fmassa/received_events","type":"User","site_admin":false},"created_at":"2015-10-19T04:13:50Z","updated_at":"2015-10-19T04:13:50Z","body":"Hi Sergey,\r\n\r\nOk, I can rename `scale`, I just took the name from `inn`.\r\n\r\nRegarding memory requirements, this PR only needs a buffer of the size of the `input` to store the `scale`, and the other buffers in `updateGradInput` doesn't require much memory, on the order of one element of `input`. I hesitated about adding a bigger buffer which would avoid having a for loop in the backward for the batch elements though...\r\n\r\nI compared this implementation with `inn` and they seem to be equivalent.\r\n\r\nThe problem at the moment is that this version is incredibly slow, 700x slower than `inn` version on GPU forward, much slower on backward. I tried to optimize it a bit by removing the for loop on forward and replacing by a matrix multiplication, see [here](https://github.com/fmassa/nn/commit/0c2e9e36e9127e140b3edffe1ad0fd577f4131eb), but on CPU it's slower, but may be faster on GPU. (Actually, at least on my machine on CPU, most of the time is spent doing `pow(-beta)`. I found out that `pow` is super slow on CPU for `beta` different than 0,1,2, and replacing it by `log(x):mul(-beta):exp())` brings a speedup of 80% compared to simply doing `pow(x,-beta)`...)"}}