{"action":"closed","number":484,"pull_request":{"url":"https://api.github.com/repos/dmlc/mxnet/pulls/484","id":49702798,"html_url":"https://github.com/dmlc/mxnet/pull/484","diff_url":"https://github.com/dmlc/mxnet/pull/484.diff","patch_url":"https://github.com/dmlc/mxnet/pull/484.patch","issue_url":"https://api.github.com/repos/dmlc/mxnet/issues/484","number":484,"state":"closed","locked":false,"title":"modified gradient clip norm & learning rate policy","user":{"login":"yang1fan2","id":4648013,"avatar_url":"https://avatars.githubusercontent.com/u/4648013?v=3","gravatar_id":"","url":"https://api.github.com/users/yang1fan2","html_url":"https://github.com/yang1fan2","followers_url":"https://api.github.com/users/yang1fan2/followers","following_url":"https://api.github.com/users/yang1fan2/following{/other_user}","gists_url":"https://api.github.com/users/yang1fan2/gists{/gist_id}","starred_url":"https://api.github.com/users/yang1fan2/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yang1fan2/subscriptions","organizations_url":"https://api.github.com/users/yang1fan2/orgs","repos_url":"https://api.github.com/users/yang1fan2/repos","events_url":"https://api.github.com/users/yang1fan2/events{/privacy}","received_events_url":"https://api.github.com/users/yang1fan2/received_events","type":"User","site_admin":false},"body":"Hi\r\n\r\nMy mxnet rnn modification achieved 103.7 on validation.\r\nThere are three modifications:\r\n1. add dropout\r\n2. Learning rate is initially set to 1.0 and halved if the perplexity does not decrease by more than 1.0 on the validation set after an epoch.\r\n3. clip the norm of the gradients (normalized by minibatch size) at 5\r\n\r\nEpoch [32732] Train: NLL=4.494, Perp=89.517\r\nEpoch [32760] Train: NLL=4.496, Perp=89.678\r\nEpoch [32788] Train: NLL=4.498, Perp=89.866\r\nEpoch [32816] Train: NLL=4.501, Perp=90.100\r\nEpoch [32844] Train: NLL=4.501, Perp=90.082\r\nEpoch [32872] Train: NLL=4.503, Perp=90.284\r\nEpoch [32900] Train: NLL=4.500, Perp=90.008\r\nEpoch [32928] Train: NLL=4.494, Perp=89.483\r\nEpoch [32956] Train: NLL=4.488, Perp=88.981\r\nEpoch [32984] Train: NLL=4.490, Perp=89.098\r\nEpoch [33012] Train: NLL=4.487, Perp=88.852\r\nEpoch [33040] Train: NLL=4.484, Perp=88.623\r\nEpoch [33068] Train: NLL=4.478, Perp=88.091\r\nEpoch [33096] Train: NLL=4.477, Perp=87.950\r\nEpoch [33124] Train: NLL=4.474, Perp=87.699\r\nEpoch [33152] Train: NLL=4.476, Perp=87.855\r\nIter [24] Train: Time: 266.847 sec, NLL=4.478, Perp=88.077\r\nIter [24] Val: NLL=4.642, Perp=103.704\r\nReset learning rate to 0.00195312\r\n\r\n\r\nPlease refer to https://github.com/yoonkim/lstm-char-cnn\r\nThanks;\r\n","created_at":"2015-11-04T15:30:54Z","updated_at":"2015-11-06T19:45:42Z","closed_at":"2015-11-06T19:45:42Z","merged_at":"2015-11-06T19:45:42Z","merge_commit_sha":"6528eefc3e36a76442bdfaab33ccfe92fe35b9c5","assignee":{"login":"antinucleon","id":1258974,"avatar_url":"https://avatars.githubusercontent.com/u/1258974?v=3","gravatar_id":"","url":"https://api.github.com/users/antinucleon","html_url":"https://github.com/antinucleon","followers_url":"https://api.github.com/users/antinucleon/followers","following_url":"https://api.github.com/users/antinucleon/following{/other_user}","gists_url":"https://api.github.com/users/antinucleon/gists{/gist_id}","starred_url":"https://api.github.com/users/antinucleon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/antinucleon/subscriptions","organizations_url":"https://api.github.com/users/antinucleon/orgs","repos_url":"https://api.github.com/users/antinucleon/repos","events_url":"https://api.github.com/users/antinucleon/events{/privacy}","received_events_url":"https://api.github.com/users/antinucleon/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/dmlc/mxnet/pulls/484/commits","review_comments_url":"https://api.github.com/repos/dmlc/mxnet/pulls/484/comments","review_comment_url":"https://api.github.com/repos/dmlc/mxnet/pulls/comments{/number}","comments_url":"https://api.github.com/repos/dmlc/mxnet/issues/484/comments","statuses_url":"https://api.github.com/repos/dmlc/mxnet/statuses/431098ebfa1e42fc8d657088eb14334e2bae206e","head":{"label":"yang1fan2:rnn","ref":"rnn","sha":"431098ebfa1e42fc8d657088eb14334e2bae206e","user":{"login":"yang1fan2","id":4648013,"avatar_url":"https://avatars.githubusercontent.com/u/4648013?v=3","gravatar_id":"","url":"https://api.github.com/users/yang1fan2","html_url":"https://github.com/yang1fan2","followers_url":"https://api.github.com/users/yang1fan2/followers","following_url":"https://api.github.com/users/yang1fan2/following{/other_user}","gists_url":"https://api.github.com/users/yang1fan2/gists{/gist_id}","starred_url":"https://api.github.com/users/yang1fan2/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yang1fan2/subscriptions","organizations_url":"https://api.github.com/users/yang1fan2/orgs","repos_url":"https://api.github.com/users/yang1fan2/repos","events_url":"https://api.github.com/users/yang1fan2/events{/privacy}","received_events_url":"https://api.github.com/users/yang1fan2/received_events","type":"User","site_admin":false},"repo":{"id":45032268,"name":"mxnet","full_name":"yang1fan2/mxnet","owner":{"login":"yang1fan2","id":4648013,"avatar_url":"https://avatars.githubusercontent.com/u/4648013?v=3","gravatar_id":"","url":"https://api.github.com/users/yang1fan2","html_url":"https://github.com/yang1fan2","followers_url":"https://api.github.com/users/yang1fan2/followers","following_url":"https://api.github.com/users/yang1fan2/following{/other_user}","gists_url":"https://api.github.com/users/yang1fan2/gists{/gist_id}","starred_url":"https://api.github.com/users/yang1fan2/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yang1fan2/subscriptions","organizations_url":"https://api.github.com/users/yang1fan2/orgs","repos_url":"https://api.github.com/users/yang1fan2/repos","events_url":"https://api.github.com/users/yang1fan2/events{/privacy}","received_events_url":"https://api.github.com/users/yang1fan2/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/yang1fan2/mxnet","description":" Efficient and Flexible Distributed Deep Learning Framework, for python, R, Julia and more","fork":true,"url":"https://api.github.com/repos/yang1fan2/mxnet","forks_url":"https://api.github.com/repos/yang1fan2/mxnet/forks","keys_url":"https://api.github.com/repos/yang1fan2/mxnet/keys{/key_id}","collaborators_url":"https://api.github.com/repos/yang1fan2/mxnet/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/yang1fan2/mxnet/teams","hooks_url":"https://api.github.com/repos/yang1fan2/mxnet/hooks","issue_events_url":"https://api.github.com/repos/yang1fan2/mxnet/issues/events{/number}","events_url":"https://api.github.com/repos/yang1fan2/mxnet/events","assignees_url":"https://api.github.com/repos/yang1fan2/mxnet/assignees{/user}","branches_url":"https://api.github.com/repos/yang1fan2/mxnet/branches{/branch}","tags_url":"https://api.github.com/repos/yang1fan2/mxnet/tags","blobs_url":"https://api.github.com/repos/yang1fan2/mxnet/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/yang1fan2/mxnet/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/yang1fan2/mxnet/git/refs{/sha}","trees_url":"https://api.github.com/repos/yang1fan2/mxnet/git/trees{/sha}","statuses_url":"https://api.github.com/repos/yang1fan2/mxnet/statuses/{sha}","languages_url":"https://api.github.com/repos/yang1fan2/mxnet/languages","stargazers_url":"https://api.github.com/repos/yang1fan2/mxnet/stargazers","contributors_url":"https://api.github.com/repos/yang1fan2/mxnet/contributors","subscribers_url":"https://api.github.com/repos/yang1fan2/mxnet/subscribers","subscription_url":"https://api.github.com/repos/yang1fan2/mxnet/subscription","commits_url":"https://api.github.com/repos/yang1fan2/mxnet/commits{/sha}","git_commits_url":"https://api.github.com/repos/yang1fan2/mxnet/git/commits{/sha}","comments_url":"https://api.github.com/repos/yang1fan2/mxnet/comments{/number}","issue_comment_url":"https://api.github.com/repos/yang1fan2/mxnet/issues/comments{/number}","contents_url":"https://api.github.com/repos/yang1fan2/mxnet/contents/{+path}","compare_url":"https://api.github.com/repos/yang1fan2/mxnet/compare/{base}...{head}","merges_url":"https://api.github.com/repos/yang1fan2/mxnet/merges","archive_url":"https://api.github.com/repos/yang1fan2/mxnet/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/yang1fan2/mxnet/downloads","issues_url":"https://api.github.com/repos/yang1fan2/mxnet/issues{/number}","pulls_url":"https://api.github.com/repos/yang1fan2/mxnet/pulls{/number}","milestones_url":"https://api.github.com/repos/yang1fan2/mxnet/milestones{/number}","notifications_url":"https://api.github.com/repos/yang1fan2/mxnet/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/yang1fan2/mxnet/labels{/name}","releases_url":"https://api.github.com/repos/yang1fan2/mxnet/releases{/id}","created_at":"2015-10-27T10:06:40Z","updated_at":"2015-10-27T10:06:43Z","pushed_at":"2015-11-05T06:45:03Z","git_url":"git://github.com/yang1fan2/mxnet.git","ssh_url":"git@github.com:yang1fan2/mxnet.git","clone_url":"https://github.com/yang1fan2/mxnet.git","svn_url":"https://github.com/yang1fan2/mxnet","homepage":"http://mxnet.rtfd.org","size":3033,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"dmlc:master","ref":"master","sha":"42a46fd0a762f5db1061dbcba425235c3f23c441","user":{"login":"dmlc","id":11508361,"avatar_url":"https://avatars.githubusercontent.com/u/11508361?v=3","gravatar_id":"","url":"https://api.github.com/users/dmlc","html_url":"https://github.com/dmlc","followers_url":"https://api.github.com/users/dmlc/followers","following_url":"https://api.github.com/users/dmlc/following{/other_user}","gists_url":"https://api.github.com/users/dmlc/gists{/gist_id}","starred_url":"https://api.github.com/users/dmlc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dmlc/subscriptions","organizations_url":"https://api.github.com/users/dmlc/orgs","repos_url":"https://api.github.com/users/dmlc/repos","events_url":"https://api.github.com/users/dmlc/events{/privacy}","received_events_url":"https://api.github.com/users/dmlc/received_events","type":"Organization","site_admin":false},"repo":{"id":34864402,"name":"mxnet","full_name":"dmlc/mxnet","owner":{"login":"dmlc","id":11508361,"avatar_url":"https://avatars.githubusercontent.com/u/11508361?v=3","gravatar_id":"","url":"https://api.github.com/users/dmlc","html_url":"https://github.com/dmlc","followers_url":"https://api.github.com/users/dmlc/followers","following_url":"https://api.github.com/users/dmlc/following{/other_user}","gists_url":"https://api.github.com/users/dmlc/gists{/gist_id}","starred_url":"https://api.github.com/users/dmlc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dmlc/subscriptions","organizations_url":"https://api.github.com/users/dmlc/orgs","repos_url":"https://api.github.com/users/dmlc/repos","events_url":"https://api.github.com/users/dmlc/events{/privacy}","received_events_url":"https://api.github.com/users/dmlc/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/dmlc/mxnet","description":"Lightweight, Efficient and Flexible Distributed Deep Learning, for python, R, Julia, Go and more","fork":false,"url":"https://api.github.com/repos/dmlc/mxnet","forks_url":"https://api.github.com/repos/dmlc/mxnet/forks","keys_url":"https://api.github.com/repos/dmlc/mxnet/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dmlc/mxnet/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dmlc/mxnet/teams","hooks_url":"https://api.github.com/repos/dmlc/mxnet/hooks","issue_events_url":"https://api.github.com/repos/dmlc/mxnet/issues/events{/number}","events_url":"https://api.github.com/repos/dmlc/mxnet/events","assignees_url":"https://api.github.com/repos/dmlc/mxnet/assignees{/user}","branches_url":"https://api.github.com/repos/dmlc/mxnet/branches{/branch}","tags_url":"https://api.github.com/repos/dmlc/mxnet/tags","blobs_url":"https://api.github.com/repos/dmlc/mxnet/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dmlc/mxnet/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dmlc/mxnet/git/refs{/sha}","trees_url":"https://api.github.com/repos/dmlc/mxnet/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dmlc/mxnet/statuses/{sha}","languages_url":"https://api.github.com/repos/dmlc/mxnet/languages","stargazers_url":"https://api.github.com/repos/dmlc/mxnet/stargazers","contributors_url":"https://api.github.com/repos/dmlc/mxnet/contributors","subscribers_url":"https://api.github.com/repos/dmlc/mxnet/subscribers","subscription_url":"https://api.github.com/repos/dmlc/mxnet/subscription","commits_url":"https://api.github.com/repos/dmlc/mxnet/commits{/sha}","git_commits_url":"https://api.github.com/repos/dmlc/mxnet/git/commits{/sha}","comments_url":"https://api.github.com/repos/dmlc/mxnet/comments{/number}","issue_comment_url":"https://api.github.com/repos/dmlc/mxnet/issues/comments{/number}","contents_url":"https://api.github.com/repos/dmlc/mxnet/contents/{+path}","compare_url":"https://api.github.com/repos/dmlc/mxnet/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dmlc/mxnet/merges","archive_url":"https://api.github.com/repos/dmlc/mxnet/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dmlc/mxnet/downloads","issues_url":"https://api.github.com/repos/dmlc/mxnet/issues{/number}","pulls_url":"https://api.github.com/repos/dmlc/mxnet/pulls{/number}","milestones_url":"https://api.github.com/repos/dmlc/mxnet/milestones{/number}","notifications_url":"https://api.github.com/repos/dmlc/mxnet/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dmlc/mxnet/labels{/name}","releases_url":"https://api.github.com/repos/dmlc/mxnet/releases{/id}","created_at":"2015-04-30T16:21:15Z","updated_at":"2015-11-06T18:27:33Z","pushed_at":"2015-11-06T19:45:42Z","git_url":"git://github.com/dmlc/mxnet.git","ssh_url":"git@github.com:dmlc/mxnet.git","clone_url":"https://github.com/dmlc/mxnet.git","svn_url":"https://github.com/dmlc/mxnet","homepage":"http://mxnet.rtfd.org","size":7226,"stargazers_count":814,"watchers_count":814,"language":"C++","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":239,"mirror_url":null,"open_issues_count":50,"forks":239,"open_issues":50,"watchers":814,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/484"},"html":{"href":"https://github.com/dmlc/mxnet/pull/484"},"issue":{"href":"https://api.github.com/repos/dmlc/mxnet/issues/484"},"comments":{"href":"https://api.github.com/repos/dmlc/mxnet/issues/484/comments"},"review_comments":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/484/comments"},"review_comment":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/dmlc/mxnet/pulls/484/commits"},"statuses":{"href":"https://api.github.com/repos/dmlc/mxnet/statuses/431098ebfa1e42fc8d657088eb14334e2bae206e"}},"merged":true,"mergeable":null,"mergeable_state":"unknown","merged_by":{"login":"antinucleon","id":1258974,"avatar_url":"https://avatars.githubusercontent.com/u/1258974?v=3","gravatar_id":"","url":"https://api.github.com/users/antinucleon","html_url":"https://github.com/antinucleon","followers_url":"https://api.github.com/users/antinucleon/followers","following_url":"https://api.github.com/users/antinucleon/following{/other_user}","gists_url":"https://api.github.com/users/antinucleon/gists{/gist_id}","starred_url":"https://api.github.com/users/antinucleon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/antinucleon/subscriptions","organizations_url":"https://api.github.com/users/antinucleon/orgs","repos_url":"https://api.github.com/users/antinucleon/repos","events_url":"https://api.github.com/users/antinucleon/events{/privacy}","received_events_url":"https://api.github.com/users/antinucleon/received_events","type":"User","site_admin":false},"comments":4,"review_comments":0,"commits":3,"additions":35,"deletions":22,"changed_files":2}}