{"action":"created","comment":{"url":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/comments/39588533","id":39588533,"diff_hunk":"@@ -56,14 +55,19 @@ function levenberg_marquardt(f::Function, g::Function, x0; tolX=1e-8, tolG=1e-12\n \t\t# we want to solve:\n \t\t#    argmin 0.5*||J(x)*delta_x + f(x)||^2 + lambda*||diagm(J'*J)*delta_x||^2\n \t\t# Solving for the minimum gives:\n-\t\t#    (J'*J + lambda*DtD) * delta_x == -J^T * f(x), where DtD = diagm(sum(J.^2,1))\n-\t\t# Where we have used the equivalence: diagm(J'*J) = diagm(sum(J.^2, 1))\n+\t\t#    (J'*J + lambda*diagm(DtD)) * delta_x == -J^T * f(x), where DtD = sumabs2(J,1)\n+\t\t# Where we have used the equivalence: diagm(J'*J) = diagm(sumabs2(J,1))\n \t\t# It is additionally useful to bound the elements of DtD below to help\n \t\t# prevent \"parameter evaporation\".\n-\t\tDtD = diagm(Float64[max(x, MIN_DIAGONAL) for x in sum(J.^2,1)])\n-\t\tdelta_x = ( J'*J + sqrt(lambda)*DtD ) \\ ( -J'*fcur )\n+\t\tDtD = sumabs2(J, 1)\n+\t\tfor i in 1:length(DtD)\n+\t\t\tif DtD[i] < MIN_DIAGONAL\n+\t\t\t\tDtD[i] = MIN_DIAGONAL\n+\t\t\tend\n+\t\tend","path":"src/levenberg_marquardt.jl","position":51,"original_position":51,"commit_id":"408a08fb07f1a4ae60d25ef531ea3b6eb0f2ea30","original_commit_id":"408a08fb07f1a4ae60d25ef531ea3b6eb0f2ea30","user":{"login":"matthieugomez","id":6223837,"avatar_url":"https://avatars.githubusercontent.com/u/6223837?v=3","gravatar_id":"","url":"https://api.github.com/users/matthieugomez","html_url":"https://github.com/matthieugomez","followers_url":"https://api.github.com/users/matthieugomez/followers","following_url":"https://api.github.com/users/matthieugomez/following{/other_user}","gists_url":"https://api.github.com/users/matthieugomez/gists{/gist_id}","starred_url":"https://api.github.com/users/matthieugomez/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matthieugomez/subscriptions","organizations_url":"https://api.github.com/users/matthieugomez/orgs","repos_url":"https://api.github.com/users/matthieugomez/repos","events_url":"https://api.github.com/users/matthieugomez/events{/privacy}","received_events_url":"https://api.github.com/users/matthieugomez/received_events","type":"User","site_admin":false},"body":"the existing code creates two vectors while this new version creates one vector (ie the max operation is done \"in place\"). But clearly it's not a bottleneck.","created_at":"2015-09-16T02:31:19Z","updated_at":"2015-09-16T02:31:19Z","html_url":"https://github.com/JuliaOpt/Optim.jl/pull/136#discussion_r39588533","pull_request_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/136","_links":{"self":{"href":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/comments/39588533"},"html":{"href":"https://github.com/JuliaOpt/Optim.jl/pull/136#discussion_r39588533"},"pull_request":{"href":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/136"}}},"pull_request":{"url":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/136","id":45084148,"html_url":"https://github.com/JuliaOpt/Optim.jl/pull/136","diff_url":"https://github.com/JuliaOpt/Optim.jl/pull/136.diff","patch_url":"https://github.com/JuliaOpt/Optim.jl/pull/136.patch","issue_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/issues/136","number":136,"state":"open","locked":false,"title":"changes in levenberg_marquardt","user":{"login":"matthieugomez","id":6223837,"avatar_url":"https://avatars.githubusercontent.com/u/6223837?v=3","gravatar_id":"","url":"https://api.github.com/users/matthieugomez","html_url":"https://github.com/matthieugomez","followers_url":"https://api.github.com/users/matthieugomez/followers","following_url":"https://api.github.com/users/matthieugomez/following{/other_user}","gists_url":"https://api.github.com/users/matthieugomez/gists{/gist_id}","starred_url":"https://api.github.com/users/matthieugomez/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matthieugomez/subscriptions","organizations_url":"https://api.github.com/users/matthieugomez/orgs","repos_url":"https://api.github.com/users/matthieugomez/repos","events_url":"https://api.github.com/users/matthieugomez/events{/privacy}","received_events_url":"https://api.github.com/users/matthieugomez/received_events","type":"User","site_admin":false},"body":"Three changes\r\n- sse is replaced by sumabs2\r\n- sqrt(lambda) is replaced by lambda^2. I think it's a typo in the original implementation.\r\n- lambda is diminished only if rho is close to one. This seems to be the standard for levenberg_marquardt\r\n","created_at":"2015-09-15T18:23:36Z","updated_at":"2015-09-16T02:31:19Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/136/commits","review_comments_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/136/comments","review_comment_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/comments{/number}","comments_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/issues/136/comments","statuses_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/statuses/408a08fb07f1a4ae60d25ef531ea3b6eb0f2ea30","head":{"label":"matthieugomez:lm","ref":"lm","sha":"408a08fb07f1a4ae60d25ef531ea3b6eb0f2ea30","user":{"login":"matthieugomez","id":6223837,"avatar_url":"https://avatars.githubusercontent.com/u/6223837?v=3","gravatar_id":"","url":"https://api.github.com/users/matthieugomez","html_url":"https://github.com/matthieugomez","followers_url":"https://api.github.com/users/matthieugomez/followers","following_url":"https://api.github.com/users/matthieugomez/following{/other_user}","gists_url":"https://api.github.com/users/matthieugomez/gists{/gist_id}","starred_url":"https://api.github.com/users/matthieugomez/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matthieugomez/subscriptions","organizations_url":"https://api.github.com/users/matthieugomez/orgs","repos_url":"https://api.github.com/users/matthieugomez/repos","events_url":"https://api.github.com/users/matthieugomez/events{/privacy}","received_events_url":"https://api.github.com/users/matthieugomez/received_events","type":"User","site_admin":false},"repo":{"id":40022904,"name":"Optim.jl","full_name":"matthieugomez/Optim.jl","owner":{"login":"matthieugomez","id":6223837,"avatar_url":"https://avatars.githubusercontent.com/u/6223837?v=3","gravatar_id":"","url":"https://api.github.com/users/matthieugomez","html_url":"https://github.com/matthieugomez","followers_url":"https://api.github.com/users/matthieugomez/followers","following_url":"https://api.github.com/users/matthieugomez/following{/other_user}","gists_url":"https://api.github.com/users/matthieugomez/gists{/gist_id}","starred_url":"https://api.github.com/users/matthieugomez/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matthieugomez/subscriptions","organizations_url":"https://api.github.com/users/matthieugomez/orgs","repos_url":"https://api.github.com/users/matthieugomez/repos","events_url":"https://api.github.com/users/matthieugomez/events{/privacy}","received_events_url":"https://api.github.com/users/matthieugomez/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/matthieugomez/Optim.jl","description":"Optimization functions for Julia","fork":true,"url":"https://api.github.com/repos/matthieugomez/Optim.jl","forks_url":"https://api.github.com/repos/matthieugomez/Optim.jl/forks","keys_url":"https://api.github.com/repos/matthieugomez/Optim.jl/keys{/key_id}","collaborators_url":"https://api.github.com/repos/matthieugomez/Optim.jl/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/matthieugomez/Optim.jl/teams","hooks_url":"https://api.github.com/repos/matthieugomez/Optim.jl/hooks","issue_events_url":"https://api.github.com/repos/matthieugomez/Optim.jl/issues/events{/number}","events_url":"https://api.github.com/repos/matthieugomez/Optim.jl/events","assignees_url":"https://api.github.com/repos/matthieugomez/Optim.jl/assignees{/user}","branches_url":"https://api.github.com/repos/matthieugomez/Optim.jl/branches{/branch}","tags_url":"https://api.github.com/repos/matthieugomez/Optim.jl/tags","blobs_url":"https://api.github.com/repos/matthieugomez/Optim.jl/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/matthieugomez/Optim.jl/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/matthieugomez/Optim.jl/git/refs{/sha}","trees_url":"https://api.github.com/repos/matthieugomez/Optim.jl/git/trees{/sha}","statuses_url":"https://api.github.com/repos/matthieugomez/Optim.jl/statuses/{sha}","languages_url":"https://api.github.com/repos/matthieugomez/Optim.jl/languages","stargazers_url":"https://api.github.com/repos/matthieugomez/Optim.jl/stargazers","contributors_url":"https://api.github.com/repos/matthieugomez/Optim.jl/contributors","subscribers_url":"https://api.github.com/repos/matthieugomez/Optim.jl/subscribers","subscription_url":"https://api.github.com/repos/matthieugomez/Optim.jl/subscription","commits_url":"https://api.github.com/repos/matthieugomez/Optim.jl/commits{/sha}","git_commits_url":"https://api.github.com/repos/matthieugomez/Optim.jl/git/commits{/sha}","comments_url":"https://api.github.com/repos/matthieugomez/Optim.jl/comments{/number}","issue_comment_url":"https://api.github.com/repos/matthieugomez/Optim.jl/issues/comments{/number}","contents_url":"https://api.github.com/repos/matthieugomez/Optim.jl/contents/{+path}","compare_url":"https://api.github.com/repos/matthieugomez/Optim.jl/compare/{base}...{head}","merges_url":"https://api.github.com/repos/matthieugomez/Optim.jl/merges","archive_url":"https://api.github.com/repos/matthieugomez/Optim.jl/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/matthieugomez/Optim.jl/downloads","issues_url":"https://api.github.com/repos/matthieugomez/Optim.jl/issues{/number}","pulls_url":"https://api.github.com/repos/matthieugomez/Optim.jl/pulls{/number}","milestones_url":"https://api.github.com/repos/matthieugomez/Optim.jl/milestones{/number}","notifications_url":"https://api.github.com/repos/matthieugomez/Optim.jl/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/matthieugomez/Optim.jl/labels{/name}","releases_url":"https://api.github.com/repos/matthieugomez/Optim.jl/releases{/id}","created_at":"2015-07-31T20:08:03Z","updated_at":"2015-07-31T20:08:03Z","pushed_at":"2015-09-15T18:17:10Z","git_url":"git://github.com/matthieugomez/Optim.jl.git","ssh_url":"git@github.com:matthieugomez/Optim.jl.git","clone_url":"https://github.com/matthieugomez/Optim.jl.git","svn_url":"https://github.com/matthieugomez/Optim.jl","homepage":"","size":559,"stargazers_count":0,"watchers_count":0,"language":"Julia","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"JuliaOpt:master","ref":"master","sha":"561eb235510711485a954f0cdd6fdadc4647d976","user":{"login":"JuliaOpt","id":5667382,"avatar_url":"https://avatars.githubusercontent.com/u/5667382?v=3","gravatar_id":"","url":"https://api.github.com/users/JuliaOpt","html_url":"https://github.com/JuliaOpt","followers_url":"https://api.github.com/users/JuliaOpt/followers","following_url":"https://api.github.com/users/JuliaOpt/following{/other_user}","gists_url":"https://api.github.com/users/JuliaOpt/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaOpt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaOpt/subscriptions","organizations_url":"https://api.github.com/users/JuliaOpt/orgs","repos_url":"https://api.github.com/users/JuliaOpt/repos","events_url":"https://api.github.com/users/JuliaOpt/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaOpt/received_events","type":"Organization","site_admin":false},"repo":{"id":3933868,"name":"Optim.jl","full_name":"JuliaOpt/Optim.jl","owner":{"login":"JuliaOpt","id":5667382,"avatar_url":"https://avatars.githubusercontent.com/u/5667382?v=3","gravatar_id":"","url":"https://api.github.com/users/JuliaOpt","html_url":"https://github.com/JuliaOpt","followers_url":"https://api.github.com/users/JuliaOpt/followers","following_url":"https://api.github.com/users/JuliaOpt/following{/other_user}","gists_url":"https://api.github.com/users/JuliaOpt/gists{/gist_id}","starred_url":"https://api.github.com/users/JuliaOpt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JuliaOpt/subscriptions","organizations_url":"https://api.github.com/users/JuliaOpt/orgs","repos_url":"https://api.github.com/users/JuliaOpt/repos","events_url":"https://api.github.com/users/JuliaOpt/events{/privacy}","received_events_url":"https://api.github.com/users/JuliaOpt/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/JuliaOpt/Optim.jl","description":"Optimization functions for Julia","fork":false,"url":"https://api.github.com/repos/JuliaOpt/Optim.jl","forks_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/forks","keys_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/keys{/key_id}","collaborators_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/teams","hooks_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/hooks","issue_events_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/issues/events{/number}","events_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/events","assignees_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/assignees{/user}","branches_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/branches{/branch}","tags_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/tags","blobs_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/git/refs{/sha}","trees_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/git/trees{/sha}","statuses_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/statuses/{sha}","languages_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/languages","stargazers_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/stargazers","contributors_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/contributors","subscribers_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/subscribers","subscription_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/subscription","commits_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/commits{/sha}","git_commits_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/git/commits{/sha}","comments_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/comments{/number}","issue_comment_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/issues/comments{/number}","contents_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/contents/{+path}","compare_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/compare/{base}...{head}","merges_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/merges","archive_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/downloads","issues_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/issues{/number}","pulls_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls{/number}","milestones_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/milestones{/number}","notifications_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/labels{/name}","releases_url":"https://api.github.com/repos/JuliaOpt/Optim.jl/releases{/id}","created_at":"2012-04-04T22:19:35Z","updated_at":"2015-09-01T14:06:19Z","pushed_at":"2015-09-15T18:23:36Z","git_url":"git://github.com/JuliaOpt/Optim.jl.git","ssh_url":"git@github.com:JuliaOpt/Optim.jl.git","clone_url":"https://github.com/JuliaOpt/Optim.jl.git","svn_url":"https://github.com/JuliaOpt/Optim.jl","homepage":"","size":1284,"stargazers_count":124,"watchers_count":124,"language":"Julia","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":49,"mirror_url":null,"open_issues_count":29,"forks":49,"open_issues":29,"watchers":124,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/136"},"html":{"href":"https://github.com/JuliaOpt/Optim.jl/pull/136"},"issue":{"href":"https://api.github.com/repos/JuliaOpt/Optim.jl/issues/136"},"comments":{"href":"https://api.github.com/repos/JuliaOpt/Optim.jl/issues/136/comments"},"review_comments":{"href":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/136/comments"},"review_comment":{"href":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/JuliaOpt/Optim.jl/pulls/136/commits"},"statuses":{"href":"https://api.github.com/repos/JuliaOpt/Optim.jl/statuses/408a08fb07f1a4ae60d25ef531ea3b6eb0f2ea30"}}}}