{"action":"closed","issue":{"url":"https://api.github.com/repos/Theano/Theano/issues/3006","labels_url":"https://api.github.com/repos/Theano/Theano/issues/3006/labels{/name}","comments_url":"https://api.github.com/repos/Theano/Theano/issues/3006/comments","events_url":"https://api.github.com/repos/Theano/Theano/issues/3006/events","html_url":"https://github.com/Theano/Theano/issues/3006","id":85744044,"number":3006,"title":"Gradient of constant evaluates to NaN","user":{"login":"xeray","id":4749504,"avatar_url":"https://avatars.githubusercontent.com/u/4749504?v=3","gravatar_id":"","url":"https://api.github.com/users/xeray","html_url":"https://github.com/xeray","followers_url":"https://api.github.com/users/xeray/followers","following_url":"https://api.github.com/users/xeray/following{/other_user}","gists_url":"https://api.github.com/users/xeray/gists{/gist_id}","starred_url":"https://api.github.com/users/xeray/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xeray/subscriptions","organizations_url":"https://api.github.com/users/xeray/orgs","repos_url":"https://api.github.com/users/xeray/repos","events_url":"https://api.github.com/users/xeray/events{/privacy}","received_events_url":"https://api.github.com/users/xeray/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-06-06T12:06:16Z","updated_at":"2015-06-10T16:41:08Z","closed_at":"2015-06-10T16:41:08Z","body":"Hello,\r\n\r\nI have observed that taking the gradient of a constant w.r.t to parameters gives NaN for the gradient of those parameters, where I expect it to be zero. I know, taking the gradient of a constant is sort of a dumb use case (it arised to me as a temporary fix for a division by 0 which I should have avoided by other means anyway). Nonetheless - if at least for the sake of mathematical consistency - it would be nice to fix it to evalutate to 0 or raise at an exception that users will know they're doing something pointless.\r\n\r\nI have some exemplary code that reproduces the behaviour (run on GPU with fastmath = True):\r\n\r\n```\r\nimport numpy as np\r\nimport theano\r\nimport theano.tensor as T\r\n\r\n\r\nW = theano.shared(np.asarray(np.random.normal(0, 0.1, size=(10,10)), dtype=np.float32), borrow=True, name='W')\r\nx = T.fvector()\r\ny = T.fvector()\r\n\r\nout_ = T.dot(x,W)\r\nerror_ = T.square(out_ - y).sum()\r\nerror_ = error_ / 0\r\nerror_      = T.where(T.isnan(error_), 0, error_)\r\nerror_      = T.where(T.isinf(error_), 0, error_)\r\ngrad_  = T.grad(error_, W)\r\n\r\nfoo = theano.function([x,y], [out_, error_, grad_])\r\nd = np.ones(10, dtype=np.float32)\r\nl = np.ones(10, dtype=np.float32)\r\nout, error, grad = foo(d,l)\r\ngrad = np.asarray(grad)\r\nprint error, grad\r\n```\r\n\r\nBtw, setting the error_ to zero by multiplying 0 works however:\r\n\r\n```\r\nout_ = T.dot(x,W)\r\nerror_ = T.square(out_ - y).sum()\r\nerror_ = error_ * 0\r\ngrad_  = T.grad(error_, W)\r\n\r\nfoo = theano.function([x,y], [out_, error_, grad_])\r\nd = np.ones(10, dtype=np.float32)\r\nl = np.ones(10, dtype=np.float32)\r\nout, error, grad = foo(d,l)\r\ngrad = np.asarray(grad)\r\nprint error, grad\r\n```\r\n\r\nCheers\r\nMarius"}}