{"action":"opened","issue":{"url":"https://api.github.com/repos/BVLC/caffe/issues/3242","labels_url":"https://api.github.com/repos/BVLC/caffe/issues/3242/labels{/name}","comments_url":"https://api.github.com/repos/BVLC/caffe/issues/3242/comments","events_url":"https://api.github.com/repos/BVLC/caffe/issues/3242/events","html_url":"https://github.com/BVLC/caffe/issues/3242","id":113094435,"number":3242,"title":"Normalization for convolution layer weights","user":{"login":"tarikarici","id":8152055,"avatar_url":"https://avatars.githubusercontent.com/u/8152055?v=3","gravatar_id":"","url":"https://api.github.com/users/tarikarici","html_url":"https://github.com/tarikarici","followers_url":"https://api.github.com/users/tarikarici/followers","following_url":"https://api.github.com/users/tarikarici/following{/other_user}","gists_url":"https://api.github.com/users/tarikarici/gists{/gist_id}","starred_url":"https://api.github.com/users/tarikarici/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tarikarici/subscriptions","organizations_url":"https://api.github.com/users/tarikarici/orgs","repos_url":"https://api.github.com/users/tarikarici/repos","events_url":"https://api.github.com/users/tarikarici/events{/privacy}","received_events_url":"https://api.github.com/users/tarikarici/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-10-23T20:33:55Z","updated_at":"2015-10-23T20:33:55Z","closed_at":null,"body":"I have noticed that gradient w.r.t convolution layer parameters are summed over all pixel multiplications of previous layer feature image and next layer gradient image. However, feature image sizes (hidden layer images) change from layer to layer. \r\n\r\nSo the gradient is a sum of number of pixels in the gradient image, without normalization. So, effectively, convolution layer weight gradients are used in stochastic gradient solver using varying learning rates. Therefore, I think in the backward methods a normalization using number of pixels in the gradient image needs be done.\r\n"}}