{"action":"created","comment":{"url":"https://api.github.com/repos/github/git-lfs/pulls/comments/42719606","id":42719606,"diff_hunk":"@@ -0,0 +1,46 @@\n+package lfs\n+\n+import (\n+\t\"os\"\n+\t\"path/filepath\"\n+\t\"strings\"\n+\t\"time\"\n+\n+\t\"github.com/github/git-lfs/vendor/_nuts/github.com/rubyist/tracerx\"\n+)\n+\n+func ClearTempObjects() {\n+\tfilepath.Walk(LocalObjectTempDir, func(path string, info os.FileInfo, err error) error {\n+\t\tif shouldDeleteTempObject(path, info) {\n+\t\t\treturn os.RemoveAll(path)\n+\t\t}\n+\n+\t\treturn err\n+\t})\n+}\n+\n+func shouldDeleteTempObject(path string, info os.FileInfo) bool {\n+\tif info == nil || info.IsDir() {\n+\t\treturn false\n+\t}\n+\n+\tbase := filepath.Base(path)\n+\tparts := strings.SplitN(base, \"-\", 2)\n+\toid := parts[0]\n+\tif len(parts) < 2 || len(oid) != 64 {\n+\t\ttracerx.Printf(\"Removing invalid tmp object file: %s\", path)\n+\t\treturn true\n+\t}\n+\n+\tif FileExists(localMediaPathNoCreate(oid)) {\n+\t\ttracerx.Printf(\"Removing existing tmp object file: %s\", path)\n+\t\treturn true\n+\t}\n+\n+\tif time.Since(info.ModTime()) > time.Hour {","path":"lfs/localstorage.go","position":40,"original_position":40,"commit_id":"7416550bc93bc9158d2ba003b4573b5898440d23","original_commit_id":"7416550bc93bc9158d2ba003b4573b5898440d23","user":{"login":"sinbad","id":142735,"avatar_url":"https://avatars.githubusercontent.com/u/142735?v=3","gravatar_id":"","url":"https://api.github.com/users/sinbad","html_url":"https://github.com/sinbad","followers_url":"https://api.github.com/users/sinbad/followers","following_url":"https://api.github.com/users/sinbad/following{/other_user}","gists_url":"https://api.github.com/users/sinbad/gists{/gist_id}","starred_url":"https://api.github.com/users/sinbad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sinbad/subscriptions","organizations_url":"https://api.github.com/users/sinbad/orgs","repos_url":"https://api.github.com/users/sinbad/repos","events_url":"https://api.github.com/users/sinbad/events{/privacy}","received_events_url":"https://api.github.com/users/sinbad/received_events","type":"User","site_admin":false},"body":"Why not just delete everything? Is this to try to avoid parallel `git-lfs` instances trampling over each other? Locks would normally prevent that anyway, except for maybe that sliver of time between finishing writing and moving a file to its final location.","created_at":"2015-10-22T07:49:51Z","updated_at":"2015-10-22T07:49:51Z","html_url":"https://github.com/github/git-lfs/pull/763#discussion_r42719606","pull_request_url":"https://api.github.com/repos/github/git-lfs/pulls/763","_links":{"self":{"href":"https://api.github.com/repos/github/git-lfs/pulls/comments/42719606"},"html":{"href":"https://github.com/github/git-lfs/pull/763#discussion_r42719606"},"pull_request":{"href":"https://api.github.com/repos/github/git-lfs/pulls/763"}}},"pull_request":{"url":"https://api.github.com/repos/github/git-lfs/pulls/763","id":48262416,"html_url":"https://github.com/github/git-lfs/pull/763","diff_url":"https://github.com/github/git-lfs/pull/763.diff","patch_url":"https://github.com/github/git-lfs/pull/763.patch","issue_url":"https://api.github.com/repos/github/git-lfs/issues/763","number":763,"state":"open","locked":false,"title":"[wip] rewrite the atomic file writes","user":{"login":"technoweenie","id":21,"avatar_url":"https://avatars.githubusercontent.com/u/21?v=3","gravatar_id":"","url":"https://api.github.com/users/technoweenie","html_url":"https://github.com/technoweenie","followers_url":"https://api.github.com/users/technoweenie/followers","following_url":"https://api.github.com/users/technoweenie/following{/other_user}","gists_url":"https://api.github.com/users/technoweenie/gists{/gist_id}","starred_url":"https://api.github.com/users/technoweenie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/technoweenie/subscriptions","organizations_url":"https://api.github.com/users/technoweenie/orgs","repos_url":"https://api.github.com/users/technoweenie/repos","events_url":"https://api.github.com/users/technoweenie/events{/privacy}","received_events_url":"https://api.github.com/users/technoweenie/received_events","type":"User","site_admin":true},"body":"This is an attempt to solve #266. If `git-lfs` is exited while downloading, it leaves some `.git/lfs/objects/../../OID-temp` files, causing future downloads (`smudge`, `fetch`, or `pull`) to error out until those temp files are deleted.\r\n\r\nThis used to work using my [contentaddressable package](https://github.com/technoweenie/go-contentaddressable) which used the `O_EXCL` file mode to ensure only one process writes to a file at a time. This breaks if the process exits without letting Go's `defer` clean the file up.\r\n\r\nThis PR changes a few things. First, it drops the contentaddressable package in favor of the [atomic package](https://github.com/natefinch/atomic). While `os.Rename()` works as an atomic operation on posix systems, it doesn't on Windows. The atomic package uses Windows APIs to provide the same atomic guarantee in `atomic.ReplaceFile()`.\r\n\r\nIt also throws all temp object files into `.git/lfs/tmp/objects`, so it's easy for `git-lfs` to clean the files up on exit. Multiple `git-lfs` processes will have their own sets of temp files. It's possible that multiple `git-lfs fetch` operations will download a file multiple times. But, it should only move the file to it's final spot (`.git/lfs/objects/../../OID`) IF the temp file is written successfully and the SHA-256 signature of the contents matches the expected OID.\r\n\r\nThis SHOULD work fine on linux/mac, but needs a bit more work for Windows support.\r\n\r\nTODO:\r\n\r\n* [x] drop `contentaddressable` package\r\n* [x] Add `atomic` package, replace `os.Rename()` usage with `atomic.ReplaceFile()`.\r\n* [x] have git-lfs cleanup old temp files automatically on exit.","created_at":"2015-10-20T21:46:02Z","updated_at":"2015-10-22T07:49:52Z","closed_at":null,"merged_at":null,"merge_commit_sha":"c459af387daa0b79dabb089d80f3e8efe90b33ca","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/github/git-lfs/pulls/763/commits","review_comments_url":"https://api.github.com/repos/github/git-lfs/pulls/763/comments","review_comment_url":"https://api.github.com/repos/github/git-lfs/pulls/comments{/number}","comments_url":"https://api.github.com/repos/github/git-lfs/issues/763/comments","statuses_url":"https://api.github.com/repos/github/git-lfs/statuses/7416550bc93bc9158d2ba003b4573b5898440d23","head":{"label":"github:atomic-writes","ref":"atomic-writes","sha":"7416550bc93bc9158d2ba003b4573b5898440d23","user":{"login":"github","id":9919,"avatar_url":"https://avatars.githubusercontent.com/u/9919?v=3","gravatar_id":"","url":"https://api.github.com/users/github","html_url":"https://github.com/github","followers_url":"https://api.github.com/users/github/followers","following_url":"https://api.github.com/users/github/following{/other_user}","gists_url":"https://api.github.com/users/github/gists{/gist_id}","starred_url":"https://api.github.com/users/github/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/github/subscriptions","organizations_url":"https://api.github.com/users/github/orgs","repos_url":"https://api.github.com/users/github/repos","events_url":"https://api.github.com/users/github/events{/privacy}","received_events_url":"https://api.github.com/users/github/received_events","type":"Organization","site_admin":false},"repo":{"id":13021798,"name":"git-lfs","full_name":"github/git-lfs","owner":{"login":"github","id":9919,"avatar_url":"https://avatars.githubusercontent.com/u/9919?v=3","gravatar_id":"","url":"https://api.github.com/users/github","html_url":"https://github.com/github","followers_url":"https://api.github.com/users/github/followers","following_url":"https://api.github.com/users/github/following{/other_user}","gists_url":"https://api.github.com/users/github/gists{/gist_id}","starred_url":"https://api.github.com/users/github/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/github/subscriptions","organizations_url":"https://api.github.com/users/github/orgs","repos_url":"https://api.github.com/users/github/repos","events_url":"https://api.github.com/users/github/events{/privacy}","received_events_url":"https://api.github.com/users/github/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/github/git-lfs","description":"Git extension for versioning large files","fork":false,"url":"https://api.github.com/repos/github/git-lfs","forks_url":"https://api.github.com/repos/github/git-lfs/forks","keys_url":"https://api.github.com/repos/github/git-lfs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/github/git-lfs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/github/git-lfs/teams","hooks_url":"https://api.github.com/repos/github/git-lfs/hooks","issue_events_url":"https://api.github.com/repos/github/git-lfs/issues/events{/number}","events_url":"https://api.github.com/repos/github/git-lfs/events","assignees_url":"https://api.github.com/repos/github/git-lfs/assignees{/user}","branches_url":"https://api.github.com/repos/github/git-lfs/branches{/branch}","tags_url":"https://api.github.com/repos/github/git-lfs/tags","blobs_url":"https://api.github.com/repos/github/git-lfs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/github/git-lfs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/github/git-lfs/git/refs{/sha}","trees_url":"https://api.github.com/repos/github/git-lfs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/github/git-lfs/statuses/{sha}","languages_url":"https://api.github.com/repos/github/git-lfs/languages","stargazers_url":"https://api.github.com/repos/github/git-lfs/stargazers","contributors_url":"https://api.github.com/repos/github/git-lfs/contributors","subscribers_url":"https://api.github.com/repos/github/git-lfs/subscribers","subscription_url":"https://api.github.com/repos/github/git-lfs/subscription","commits_url":"https://api.github.com/repos/github/git-lfs/commits{/sha}","git_commits_url":"https://api.github.com/repos/github/git-lfs/git/commits{/sha}","comments_url":"https://api.github.com/repos/github/git-lfs/comments{/number}","issue_comment_url":"https://api.github.com/repos/github/git-lfs/issues/comments{/number}","contents_url":"https://api.github.com/repos/github/git-lfs/contents/{+path}","compare_url":"https://api.github.com/repos/github/git-lfs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/github/git-lfs/merges","archive_url":"https://api.github.com/repos/github/git-lfs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/github/git-lfs/downloads","issues_url":"https://api.github.com/repos/github/git-lfs/issues{/number}","pulls_url":"https://api.github.com/repos/github/git-lfs/pulls{/number}","milestones_url":"https://api.github.com/repos/github/git-lfs/milestones{/number}","notifications_url":"https://api.github.com/repos/github/git-lfs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/github/git-lfs/labels{/name}","releases_url":"https://api.github.com/repos/github/git-lfs/releases{/id}","created_at":"2013-09-22T22:03:28Z","updated_at":"2015-10-22T05:35:37Z","pushed_at":"2015-10-21T22:27:14Z","git_url":"git://github.com/github/git-lfs.git","ssh_url":"git@github.com:github/git-lfs.git","clone_url":"https://github.com/github/git-lfs.git","svn_url":"https://github.com/github/git-lfs","homepage":"https://git-lfs.github.com","size":13741,"stargazers_count":2869,"watchers_count":2869,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":229,"mirror_url":null,"open_issues_count":155,"forks":229,"open_issues":155,"watchers":2869,"default_branch":"master"}},"base":{"label":"github:master","ref":"master","sha":"3a673d662b63ab2cd81bf932de9ee313fe1dc736","user":{"login":"github","id":9919,"avatar_url":"https://avatars.githubusercontent.com/u/9919?v=3","gravatar_id":"","url":"https://api.github.com/users/github","html_url":"https://github.com/github","followers_url":"https://api.github.com/users/github/followers","following_url":"https://api.github.com/users/github/following{/other_user}","gists_url":"https://api.github.com/users/github/gists{/gist_id}","starred_url":"https://api.github.com/users/github/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/github/subscriptions","organizations_url":"https://api.github.com/users/github/orgs","repos_url":"https://api.github.com/users/github/repos","events_url":"https://api.github.com/users/github/events{/privacy}","received_events_url":"https://api.github.com/users/github/received_events","type":"Organization","site_admin":false},"repo":{"id":13021798,"name":"git-lfs","full_name":"github/git-lfs","owner":{"login":"github","id":9919,"avatar_url":"https://avatars.githubusercontent.com/u/9919?v=3","gravatar_id":"","url":"https://api.github.com/users/github","html_url":"https://github.com/github","followers_url":"https://api.github.com/users/github/followers","following_url":"https://api.github.com/users/github/following{/other_user}","gists_url":"https://api.github.com/users/github/gists{/gist_id}","starred_url":"https://api.github.com/users/github/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/github/subscriptions","organizations_url":"https://api.github.com/users/github/orgs","repos_url":"https://api.github.com/users/github/repos","events_url":"https://api.github.com/users/github/events{/privacy}","received_events_url":"https://api.github.com/users/github/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/github/git-lfs","description":"Git extension for versioning large files","fork":false,"url":"https://api.github.com/repos/github/git-lfs","forks_url":"https://api.github.com/repos/github/git-lfs/forks","keys_url":"https://api.github.com/repos/github/git-lfs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/github/git-lfs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/github/git-lfs/teams","hooks_url":"https://api.github.com/repos/github/git-lfs/hooks","issue_events_url":"https://api.github.com/repos/github/git-lfs/issues/events{/number}","events_url":"https://api.github.com/repos/github/git-lfs/events","assignees_url":"https://api.github.com/repos/github/git-lfs/assignees{/user}","branches_url":"https://api.github.com/repos/github/git-lfs/branches{/branch}","tags_url":"https://api.github.com/repos/github/git-lfs/tags","blobs_url":"https://api.github.com/repos/github/git-lfs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/github/git-lfs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/github/git-lfs/git/refs{/sha}","trees_url":"https://api.github.com/repos/github/git-lfs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/github/git-lfs/statuses/{sha}","languages_url":"https://api.github.com/repos/github/git-lfs/languages","stargazers_url":"https://api.github.com/repos/github/git-lfs/stargazers","contributors_url":"https://api.github.com/repos/github/git-lfs/contributors","subscribers_url":"https://api.github.com/repos/github/git-lfs/subscribers","subscription_url":"https://api.github.com/repos/github/git-lfs/subscription","commits_url":"https://api.github.com/repos/github/git-lfs/commits{/sha}","git_commits_url":"https://api.github.com/repos/github/git-lfs/git/commits{/sha}","comments_url":"https://api.github.com/repos/github/git-lfs/comments{/number}","issue_comment_url":"https://api.github.com/repos/github/git-lfs/issues/comments{/number}","contents_url":"https://api.github.com/repos/github/git-lfs/contents/{+path}","compare_url":"https://api.github.com/repos/github/git-lfs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/github/git-lfs/merges","archive_url":"https://api.github.com/repos/github/git-lfs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/github/git-lfs/downloads","issues_url":"https://api.github.com/repos/github/git-lfs/issues{/number}","pulls_url":"https://api.github.com/repos/github/git-lfs/pulls{/number}","milestones_url":"https://api.github.com/repos/github/git-lfs/milestones{/number}","notifications_url":"https://api.github.com/repos/github/git-lfs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/github/git-lfs/labels{/name}","releases_url":"https://api.github.com/repos/github/git-lfs/releases{/id}","created_at":"2013-09-22T22:03:28Z","updated_at":"2015-10-22T05:35:37Z","pushed_at":"2015-10-21T22:27:14Z","git_url":"git://github.com/github/git-lfs.git","ssh_url":"git@github.com:github/git-lfs.git","clone_url":"https://github.com/github/git-lfs.git","svn_url":"https://github.com/github/git-lfs","homepage":"https://git-lfs.github.com","size":13741,"stargazers_count":2869,"watchers_count":2869,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":229,"mirror_url":null,"open_issues_count":155,"forks":229,"open_issues":155,"watchers":2869,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/github/git-lfs/pulls/763"},"html":{"href":"https://github.com/github/git-lfs/pull/763"},"issue":{"href":"https://api.github.com/repos/github/git-lfs/issues/763"},"comments":{"href":"https://api.github.com/repos/github/git-lfs/issues/763/comments"},"review_comments":{"href":"https://api.github.com/repos/github/git-lfs/pulls/763/comments"},"review_comment":{"href":"https://api.github.com/repos/github/git-lfs/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/github/git-lfs/pulls/763/commits"},"statuses":{"href":"https://api.github.com/repos/github/git-lfs/statuses/7416550bc93bc9158d2ba003b4573b5898440d23"}}}}