{"action":"created","issue":{"url":"https://api.github.com/repos/Russell91/nlpcaffe/issues/4","labels_url":"https://api.github.com/repos/Russell91/nlpcaffe/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/Russell91/nlpcaffe/issues/4/comments","events_url":"https://api.github.com/repos/Russell91/nlpcaffe/issues/4/events","html_url":"https://github.com/Russell91/nlpcaffe/issues/4","id":69491799,"number":4,"title":"what's the meaning of \"scaling down gradients\" in training log?","user":{"login":"imaglib","id":11958317,"avatar_url":"https://avatars.githubusercontent.com/u/11958317?v=3","gravatar_id":"","url":"https://api.github.com/users/imaglib","html_url":"https://github.com/imaglib","followers_url":"https://api.github.com/users/imaglib/followers","following_url":"https://api.github.com/users/imaglib/following{/other_user}","gists_url":"https://api.github.com/users/imaglib/gists{/gist_id}","starred_url":"https://api.github.com/users/imaglib/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imaglib/subscriptions","organizations_url":"https://api.github.com/users/imaglib/orgs","repos_url":"https://api.github.com/users/imaglib/repos","events_url":"https://api.github.com/users/imaglib/events{/privacy}","received_events_url":"https://api.github.com/users/imaglib/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-04-20T02:23:20Z","updated_at":"2015-04-20T05:09:40Z","closed_at":"2015-04-20T03:19:55Z","body":"Hi Russell:\r\n  During the training , there are the std log \" Gradient clipping: scaling down gradients (L2 norm 0.243775 > 0.24) by scale factor 0.984513\".  The 0.24 is the threshold of gradients? Why set this value , empirical value? RNN has drawback of gradient vanishing, Why there is bigger value ?"},"comment":{"url":"https://api.github.com/repos/Russell91/nlpcaffe/issues/comments/94358206","html_url":"https://github.com/Russell91/nlpcaffe/issues/4#issuecomment-94358206","issue_url":"https://api.github.com/repos/Russell91/nlpcaffe/issues/4","id":94358206,"user":{"login":"imaglib","id":11958317,"avatar_url":"https://avatars.githubusercontent.com/u/11958317?v=3","gravatar_id":"","url":"https://api.github.com/users/imaglib","html_url":"https://github.com/imaglib","followers_url":"https://api.github.com/users/imaglib/followers","following_url":"https://api.github.com/users/imaglib/following{/other_user}","gists_url":"https://api.github.com/users/imaglib/gists{/gist_id}","starred_url":"https://api.github.com/users/imaglib/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imaglib/subscriptions","organizations_url":"https://api.github.com/users/imaglib/orgs","repos_url":"https://api.github.com/users/imaglib/repos","events_url":"https://api.github.com/users/imaglib/events{/privacy}","received_events_url":"https://api.github.com/users/imaglib/received_events","type":"User","site_admin":false},"created_at":"2015-04-20T05:09:40Z","updated_at":"2015-04-20T05:09:40Z","body":"I mean the RNN has drawback of gradient vanishing, so i think the gradient should be samller, why use the gradient clipling to limit the gradient?"}}