{"action":"created","comment":{"url":"https://api.github.com/repos/docker/docker/pulls/comments/34069213","id":34069213,"diff_hunk":"@@ -485,10 +516,51 @@ func (graph *Graph) RawJSON(id string) ([]byte, error) {\n }\n \n func jsonPath(root string) string {\n-\treturn filepath.Join(root, \"json\")\n+\treturn filepath.Join(root, jsonFileName)\n }\n \n // TarLayer returns a tar archive of the image's filesystem layer.\n func (graph *Graph) TarLayer(img *image.Image) (arch archive.Archive, err error) {\n-\treturn graph.driver.Diff(img.ID, img.Parent)\n+\troot := graph.imageRoot(img.ID)\n+\tmFileName := filepath.Join(root, tarDataFileName)\n+\tmf, err := os.Open(mFileName)\n+\tif err != nil {\n+\t\tif !os.IsNotExist(err) {\n+\t\t\tlogrus.Errorf(\"failed to open %q: %s\", mFileName, err)\n+\t\t}\n+\t\tlogrus.Debugf(\"[graph] TarLayer with traditional differ: %s\", img.ID)\n+\t\treturn graph.driver.Diff(img.ID, img.Parent)\n+\t}\n+\tpR, pW := io.Pipe()\n+\t// this will need to be in a goroutine, as we are returning the stream of a","path":"graph/graph.go","position":116,"original_position":116,"commit_id":"58bb493627dc01050b5d4365c1e089c9547c7907","original_commit_id":"58bb493627dc01050b5d4365c1e089c9547c7907","user":{"login":"stevvooe","id":120601,"avatar_url":"https://avatars.githubusercontent.com/u/120601?v=3","gravatar_id":"","url":"https://api.github.com/users/stevvooe","html_url":"https://github.com/stevvooe","followers_url":"https://api.github.com/users/stevvooe/followers","following_url":"https://api.github.com/users/stevvooe/following{/other_user}","gists_url":"https://api.github.com/users/stevvooe/gists{/gist_id}","starred_url":"https://api.github.com/users/stevvooe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stevvooe/subscriptions","organizations_url":"https://api.github.com/users/stevvooe/orgs","repos_url":"https://api.github.com/users/stevvooe/repos","events_url":"https://api.github.com/users/stevvooe/events{/privacy}","received_events_url":"https://api.github.com/users/stevvooe/received_events","type":"User","site_admin":false},"body":"Is there any way to capture this abstraction in the tar-split library? If metaUnpacker and fileGetter managed their constituent resources, mfz and fsLayer, as `io.ReadCloser`, this would be unnecessary.","created_at":"2015-07-07T17:54:21Z","updated_at":"2015-07-07T17:54:21Z","html_url":"https://github.com/docker/docker/pull/14067#discussion_r34069213","pull_request_url":"https://api.github.com/repos/docker/docker/pulls/14067","_links":{"self":{"href":"https://api.github.com/repos/docker/docker/pulls/comments/34069213"},"html":{"href":"https://github.com/docker/docker/pull/14067#discussion_r34069213"},"pull_request":{"href":"https://api.github.com/repos/docker/docker/pulls/14067"}}},"pull_request":{"url":"https://api.github.com/repos/docker/docker/pulls/14067","id":38177256,"html_url":"https://github.com/docker/docker/pull/14067","diff_url":"https://github.com/docker/docker/pull/14067.diff","patch_url":"https://github.com/docker/docker/pull/14067.patch","issue_url":"https://api.github.com/repos/docker/docker/issues/14067","number":14067,"state":"open","locked":false,"title":"graph: layer tar disassembly and reassembly","user":{"login":"vbatts","id":67049,"avatar_url":"https://avatars.githubusercontent.com/u/67049?v=3","gravatar_id":"","url":"https://api.github.com/users/vbatts","html_url":"https://github.com/vbatts","followers_url":"https://api.github.com/users/vbatts/followers","following_url":"https://api.github.com/users/vbatts/following{/other_user}","gists_url":"https://api.github.com/users/vbatts/gists{/gist_id}","starred_url":"https://api.github.com/users/vbatts/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vbatts/subscriptions","organizations_url":"https://api.github.com/users/vbatts/orgs","repos_url":"https://api.github.com/users/vbatts/repos","events_url":"https://api.github.com/users/vbatts/events{/privacy}","received_events_url":"https://api.github.com/users/vbatts/received_events","type":"User","site_admin":false},"body":"## Overview\r\n\r\nThis pull request introduces a library ([vbatts/tar-split](https://github.com/vbatts/tar-split)) that is for inline disassembly of TAR archives. The disassembly does not do _any_ extraction, but preserves the raw bytes of headers and padding from the archive, and defers the extraction. This way validation and extraction continue to be a lockstep process.\r\n\r\nThe reassembly use this preserved archive data and the relative path of the image layer, to produce a TAR archive stream.\r\n\r\n## Changes\r\n\r\nThis change is completely transparent to existing images and installations.\r\n\r\nFor an `docker pull`, `docker load` or `docker commit` after this feature, there will be a new state file stored for the image layer (e.g. `/var/lib/docker/graph/<ID>/tar-data.json.gz`).\r\n\r\nFor existing images that do _not_ have this new state file, the tar archive produced by `docker save` or `docker push` will fallback to the traditional `graphdriver.Diff` as they have been.\r\n\r\n## Benefits\r\n\r\nThe benefit of this feature is that rather than _hoping_ the `graph.TarLayer` produced is deterministic, the tar archive will be reassembled from the raw bytes of the original archive.\r\n\r\nPresently the issue exists that an image pulled from a repo like the Docker hub, and then pushed to a local registry may likely have a new digests.\r\n\r\n\r\n\r\n/cc @ncdc @stevvooe @dmcgowan @jlhawn @NathanMcCauley @crosbymichael ","created_at":"2015-06-21T01:15:34Z","updated_at":"2015-07-07T17:54:21Z","closed_at":null,"merged_at":null,"merge_commit_sha":"505f7bf77c024cc7acb25546365f0d6de0e11eb0","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/docker/docker/pulls/14067/commits","review_comments_url":"https://api.github.com/repos/docker/docker/pulls/14067/comments","review_comment_url":"https://api.github.com/repos/docker/docker/pulls/comments{/number}","comments_url":"https://api.github.com/repos/docker/docker/issues/14067/comments","statuses_url":"https://api.github.com/repos/docker/docker/statuses/58bb493627dc01050b5d4365c1e089c9547c7907","head":{"label":"vbatts:vbatts-tar-split","ref":"vbatts-tar-split","sha":"58bb493627dc01050b5d4365c1e089c9547c7907","user":{"login":"vbatts","id":67049,"avatar_url":"https://avatars.githubusercontent.com/u/67049?v=3","gravatar_id":"","url":"https://api.github.com/users/vbatts","html_url":"https://github.com/vbatts","followers_url":"https://api.github.com/users/vbatts/followers","following_url":"https://api.github.com/users/vbatts/following{/other_user}","gists_url":"https://api.github.com/users/vbatts/gists{/gist_id}","starred_url":"https://api.github.com/users/vbatts/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vbatts/subscriptions","organizations_url":"https://api.github.com/users/vbatts/orgs","repos_url":"https://api.github.com/users/vbatts/repos","events_url":"https://api.github.com/users/vbatts/events{/privacy}","received_events_url":"https://api.github.com/users/vbatts/received_events","type":"User","site_admin":false},"repo":{"id":14430080,"name":"docker","full_name":"vbatts/docker","owner":{"login":"vbatts","id":67049,"avatar_url":"https://avatars.githubusercontent.com/u/67049?v=3","gravatar_id":"","url":"https://api.github.com/users/vbatts","html_url":"https://github.com/vbatts","followers_url":"https://api.github.com/users/vbatts/followers","following_url":"https://api.github.com/users/vbatts/following{/other_user}","gists_url":"https://api.github.com/users/vbatts/gists{/gist_id}","starred_url":"https://api.github.com/users/vbatts/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vbatts/subscriptions","organizations_url":"https://api.github.com/users/vbatts/orgs","repos_url":"https://api.github.com/users/vbatts/repos","events_url":"https://api.github.com/users/vbatts/events{/privacy}","received_events_url":"https://api.github.com/users/vbatts/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/vbatts/docker","description":"Docker - the open-source application container engine","fork":true,"url":"https://api.github.com/repos/vbatts/docker","forks_url":"https://api.github.com/repos/vbatts/docker/forks","keys_url":"https://api.github.com/repos/vbatts/docker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/vbatts/docker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/vbatts/docker/teams","hooks_url":"https://api.github.com/repos/vbatts/docker/hooks","issue_events_url":"https://api.github.com/repos/vbatts/docker/issues/events{/number}","events_url":"https://api.github.com/repos/vbatts/docker/events","assignees_url":"https://api.github.com/repos/vbatts/docker/assignees{/user}","branches_url":"https://api.github.com/repos/vbatts/docker/branches{/branch}","tags_url":"https://api.github.com/repos/vbatts/docker/tags","blobs_url":"https://api.github.com/repos/vbatts/docker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/vbatts/docker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/vbatts/docker/git/refs{/sha}","trees_url":"https://api.github.com/repos/vbatts/docker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/vbatts/docker/statuses/{sha}","languages_url":"https://api.github.com/repos/vbatts/docker/languages","stargazers_url":"https://api.github.com/repos/vbatts/docker/stargazers","contributors_url":"https://api.github.com/repos/vbatts/docker/contributors","subscribers_url":"https://api.github.com/repos/vbatts/docker/subscribers","subscription_url":"https://api.github.com/repos/vbatts/docker/subscription","commits_url":"https://api.github.com/repos/vbatts/docker/commits{/sha}","git_commits_url":"https://api.github.com/repos/vbatts/docker/git/commits{/sha}","comments_url":"https://api.github.com/repos/vbatts/docker/comments{/number}","issue_comment_url":"https://api.github.com/repos/vbatts/docker/issues/comments{/number}","contents_url":"https://api.github.com/repos/vbatts/docker/contents/{+path}","compare_url":"https://api.github.com/repos/vbatts/docker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/vbatts/docker/merges","archive_url":"https://api.github.com/repos/vbatts/docker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/vbatts/docker/downloads","issues_url":"https://api.github.com/repos/vbatts/docker/issues{/number}","pulls_url":"https://api.github.com/repos/vbatts/docker/pulls{/number}","milestones_url":"https://api.github.com/repos/vbatts/docker/milestones{/number}","notifications_url":"https://api.github.com/repos/vbatts/docker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/vbatts/docker/labels{/name}","releases_url":"https://api.github.com/repos/vbatts/docker/releases{/id}","created_at":"2013-11-15T17:00:27Z","updated_at":"2015-04-08T19:38:00Z","pushed_at":"2015-07-07T15:38:27Z","git_url":"git://github.com/vbatts/docker.git","ssh_url":"git@github.com:vbatts/docker.git","clone_url":"https://github.com/vbatts/docker.git","svn_url":"https://github.com/vbatts/docker","homepage":"http://www.docker.io","size":60930,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"docker:master","ref":"master","sha":"10bbf62081ac85b17a93d79ba43eec43bebe468c","user":{"login":"docker","id":5429470,"avatar_url":"https://avatars.githubusercontent.com/u/5429470?v=3","gravatar_id":"","url":"https://api.github.com/users/docker","html_url":"https://github.com/docker","followers_url":"https://api.github.com/users/docker/followers","following_url":"https://api.github.com/users/docker/following{/other_user}","gists_url":"https://api.github.com/users/docker/gists{/gist_id}","starred_url":"https://api.github.com/users/docker/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/docker/subscriptions","organizations_url":"https://api.github.com/users/docker/orgs","repos_url":"https://api.github.com/users/docker/repos","events_url":"https://api.github.com/users/docker/events{/privacy}","received_events_url":"https://api.github.com/users/docker/received_events","type":"Organization","site_admin":false},"repo":{"id":7691631,"name":"docker","full_name":"docker/docker","owner":{"login":"docker","id":5429470,"avatar_url":"https://avatars.githubusercontent.com/u/5429470?v=3","gravatar_id":"","url":"https://api.github.com/users/docker","html_url":"https://github.com/docker","followers_url":"https://api.github.com/users/docker/followers","following_url":"https://api.github.com/users/docker/following{/other_user}","gists_url":"https://api.github.com/users/docker/gists{/gist_id}","starred_url":"https://api.github.com/users/docker/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/docker/subscriptions","organizations_url":"https://api.github.com/users/docker/orgs","repos_url":"https://api.github.com/users/docker/repos","events_url":"https://api.github.com/users/docker/events{/privacy}","received_events_url":"https://api.github.com/users/docker/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/docker/docker","description":"Docker - the open-source application container engine","fork":false,"url":"https://api.github.com/repos/docker/docker","forks_url":"https://api.github.com/repos/docker/docker/forks","keys_url":"https://api.github.com/repos/docker/docker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/docker/docker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/docker/docker/teams","hooks_url":"https://api.github.com/repos/docker/docker/hooks","issue_events_url":"https://api.github.com/repos/docker/docker/issues/events{/number}","events_url":"https://api.github.com/repos/docker/docker/events","assignees_url":"https://api.github.com/repos/docker/docker/assignees{/user}","branches_url":"https://api.github.com/repos/docker/docker/branches{/branch}","tags_url":"https://api.github.com/repos/docker/docker/tags","blobs_url":"https://api.github.com/repos/docker/docker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/docker/docker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/docker/docker/git/refs{/sha}","trees_url":"https://api.github.com/repos/docker/docker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/docker/docker/statuses/{sha}","languages_url":"https://api.github.com/repos/docker/docker/languages","stargazers_url":"https://api.github.com/repos/docker/docker/stargazers","contributors_url":"https://api.github.com/repos/docker/docker/contributors","subscribers_url":"https://api.github.com/repos/docker/docker/subscribers","subscription_url":"https://api.github.com/repos/docker/docker/subscription","commits_url":"https://api.github.com/repos/docker/docker/commits{/sha}","git_commits_url":"https://api.github.com/repos/docker/docker/git/commits{/sha}","comments_url":"https://api.github.com/repos/docker/docker/comments{/number}","issue_comment_url":"https://api.github.com/repos/docker/docker/issues/comments{/number}","contents_url":"https://api.github.com/repos/docker/docker/contents/{+path}","compare_url":"https://api.github.com/repos/docker/docker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/docker/docker/merges","archive_url":"https://api.github.com/repos/docker/docker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/docker/docker/downloads","issues_url":"https://api.github.com/repos/docker/docker/issues{/number}","pulls_url":"https://api.github.com/repos/docker/docker/pulls{/number}","milestones_url":"https://api.github.com/repos/docker/docker/milestones{/number}","notifications_url":"https://api.github.com/repos/docker/docker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/docker/docker/labels{/name}","releases_url":"https://api.github.com/repos/docker/docker/releases{/id}","created_at":"2013-01-18T18:10:57Z","updated_at":"2015-07-07T17:06:28Z","pushed_at":"2015-07-07T17:54:01Z","git_url":"git://github.com/docker/docker.git","ssh_url":"git@github.com:docker/docker.git","clone_url":"https://github.com/docker/docker.git","svn_url":"https://github.com/docker/docker","homepage":"http://www.docker.com","size":220119,"stargazers_count":23069,"watchers_count":23069,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":5662,"mirror_url":null,"open_issues_count":1121,"forks":5662,"open_issues":1121,"watchers":23069,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/docker/docker/pulls/14067"},"html":{"href":"https://github.com/docker/docker/pull/14067"},"issue":{"href":"https://api.github.com/repos/docker/docker/issues/14067"},"comments":{"href":"https://api.github.com/repos/docker/docker/issues/14067/comments"},"review_comments":{"href":"https://api.github.com/repos/docker/docker/pulls/14067/comments"},"review_comment":{"href":"https://api.github.com/repos/docker/docker/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/docker/docker/pulls/14067/commits"},"statuses":{"href":"https://api.github.com/repos/docker/docker/statuses/58bb493627dc01050b5d4365c1e089c9547c7907"}}}}