{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/1244","labels_url":"https://api.github.com/repos/fchollet/keras/issues/1244/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/1244/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/1244/events","html_url":"https://github.com/fchollet/keras/issues/1244","id":121678994,"number":1244,"title":"Loss turns into 'nan' when running on GPU","user":{"login":"lmoesch","id":16242519,"avatar_url":"https://avatars.githubusercontent.com/u/16242519?v=3","gravatar_id":"","url":"https://api.github.com/users/lmoesch","html_url":"https://github.com/lmoesch","followers_url":"https://api.github.com/users/lmoesch/followers","following_url":"https://api.github.com/users/lmoesch/following{/other_user}","gists_url":"https://api.github.com/users/lmoesch/gists{/gist_id}","starred_url":"https://api.github.com/users/lmoesch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lmoesch/subscriptions","organizations_url":"https://api.github.com/users/lmoesch/orgs","repos_url":"https://api.github.com/users/lmoesch/repos","events_url":"https://api.github.com/users/lmoesch/events{/privacy}","received_events_url":"https://api.github.com/users/lmoesch/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-12-11T10:15:30Z","updated_at":"2015-12-11T16:34:26Z","closed_at":null,"body":"Like previously stated in issue #511 Keras runs into not a number losses while training on GPU.\r\nTested this with the mnist_cnn example code aswell as with self designed conv networks. I also tried to disable cuDnn, aswell as increasing the epsilon and setting a clinorm. Nothing solved the poblem.\r\n\r\nI'm using the latest version of Theano and Keras. And SGD optimisation with categorical crossentropy.\r\n\r\nGraphics: GTX 980 Ti"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/163984222","html_url":"https://github.com/fchollet/keras/issues/1244#issuecomment-163984222","issue_url":"https://api.github.com/repos/fchollet/keras/issues/1244","id":163984222,"user":{"login":"fchollet","id":710255,"avatar_url":"https://avatars.githubusercontent.com/u/710255?v=3","gravatar_id":"","url":"https://api.github.com/users/fchollet","html_url":"https://github.com/fchollet","followers_url":"https://api.github.com/users/fchollet/followers","following_url":"https://api.github.com/users/fchollet/following{/other_user}","gists_url":"https://api.github.com/users/fchollet/gists{/gist_id}","starred_url":"https://api.github.com/users/fchollet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fchollet/subscriptions","organizations_url":"https://api.github.com/users/fchollet/orgs","repos_url":"https://api.github.com/users/fchollet/repos","events_url":"https://api.github.com/users/fchollet/events{/privacy}","received_events_url":"https://api.github.com/users/fchollet/received_events","type":"User","site_admin":false},"created_at":"2015-12-11T16:34:26Z","updated_at":"2015-12-11T16:34:26Z","body":"I'd like to identify what op is causing this issue. \r\n\r\n- post your code here.\r\n- try to use a different loss than categorical crossentropy, e.g. MSE"}}