{"action":"closed","number":52,"pull_request":{"url":"https://api.github.com/repos/polydawn/repeatr/pulls/52","id":35890297,"html_url":"https://github.com/polydawn/repeatr/pull/52","diff_url":"https://github.com/polydawn/repeatr/pull/52.diff","patch_url":"https://github.com/polydawn/repeatr/pull/52.patch","issue_url":"https://api.github.com/repos/polydawn/repeatr/issues/52","number":52,"state":"closed","locked":false,"title":"IO reboot","user":{"login":"heavenlyhash","id":627638,"avatar_url":"https://avatars.githubusercontent.com/u/627638?v=3","gravatar_id":"","url":"https://api.github.com/users/heavenlyhash","html_url":"https://github.com/heavenlyhash","followers_url":"https://api.github.com/users/heavenlyhash/followers","following_url":"https://api.github.com/users/heavenlyhash/following{/other_user}","gists_url":"https://api.github.com/users/heavenlyhash/gists{/gist_id}","starred_url":"https://api.github.com/users/heavenlyhash/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/heavenlyhash/subscriptions","organizations_url":"https://api.github.com/users/heavenlyhash/orgs","repos_url":"https://api.github.com/users/heavenlyhash/repos","events_url":"https://api.github.com/users/heavenlyhash/events{/privacy}","received_events_url":"https://api.github.com/users/heavenlyhash/received_events","type":"User","site_admin":false},"body":"Significant reorganization of IO code, covering three major objectives:\r\n\r\n- Move input and output code for a system together, because they're turning out to usually have more to do with their other half than they do with all the other inputs/outputs.\r\n- Make all major interfaces blocking instead of returning channels.\r\n- Reoriented all input data materialization code to *return* a path.  (This is to help make caching possible, and enable zero-copy systems.)\r\n- Prepare transport systems to be more plugin-ready -- the critical interfaces are now composable.\r\n\r\n...FOUR major objectives.\r\n\r\n- Get parallel assembly working, e.g. so we can start downloading assets A, B, and C in parallel, then toss the rootfs together as quickly as possible after that.\r\n\r\n... FIVE... major objectives.\r\n\r\nAnd also:\r\n\r\n- Use more typedefs to express things that should already be validated parameters.  Pass those around individually; avoid speaking of the structs in `def`, because those will move to being more the user-facing API (they make sense as a config description, but less so on internals; the number of code paths that need to see e.g. mount location and silo URI at the same time are precisely zilch).\r\n- Make URIs a list.  It's not used much yet (the current API changes mostly shim back to the prior ones, and we'll finish those migrations in a subsequent PR because this one is already way too huge), but it makes A) zero locations make sense for output scans if we just want to get a description of the data without uploading it to storage and B) plural upload locations make sense while doing a single hashing pass.\r\n- Use [Cheney's optional pattern](http://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis) to get IO systems on a path that allows more complex invocations; some now, and room to expand in the future without wrecking function signatures.\r\n\r\nSo a lot of objectives.\r\n\r\nAnd on top of all that, this contains working implementations of copy-on-write systems for filesystem assembly (as well as fallbacks to a basic cp-based system), which means if you run a job using a twelve different 30GB assets and it takes a long time to provision... if you run that job *again* it will start in around a hundredth of a second.  This feature applies to both the rootfs (e.g. your ubuntu tarball or whatever) as well as *any other inputs you specify*.\r\n\r\nSome names were picked by involvement of a dartboard and might be best considered working titles to be amended in future PRs upon better ideas evolving.\r\n\r\n:warning: contains commits from other PRs: rebasing will remove them after #51 lands -- EDIT: done\r\n\r\nThere's still a lot of to-do's to follow this -- in particular, as mentioned above, the old IO code is still just swaddled by the new interfaces, and should be fully converted -- but the diff size has gotten sufficiently large that it's desirable to land it so we can start working on incremental things again, rather than make this any more unwieldy.","created_at":"2015-05-21T00:36:31Z","updated_at":"2015-05-24T15:36:07Z","closed_at":"2015-05-24T15:36:07Z","merged_at":"2015-05-24T15:36:07Z","merge_commit_sha":"05cbc69f7f132af0ccf623479c1a0d1e6827fa5b","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/polydawn/repeatr/pulls/52/commits","review_comments_url":"https://api.github.com/repos/polydawn/repeatr/pulls/52/comments","review_comment_url":"https://api.github.com/repos/polydawn/repeatr/pulls/comments{/number}","comments_url":"https://api.github.com/repos/polydawn/repeatr/issues/52/comments","statuses_url":"https://api.github.com/repos/polydawn/repeatr/statuses/1caf5f1cef2c3734d933b24abe8d5945370c8a0b","head":{"label":"polydawn:io-reboot","ref":"io-reboot","sha":"1caf5f1cef2c3734d933b24abe8d5945370c8a0b","user":{"login":"polydawn","id":5307883,"avatar_url":"https://avatars.githubusercontent.com/u/5307883?v=3","gravatar_id":"","url":"https://api.github.com/users/polydawn","html_url":"https://github.com/polydawn","followers_url":"https://api.github.com/users/polydawn/followers","following_url":"https://api.github.com/users/polydawn/following{/other_user}","gists_url":"https://api.github.com/users/polydawn/gists{/gist_id}","starred_url":"https://api.github.com/users/polydawn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/polydawn/subscriptions","organizations_url":"https://api.github.com/users/polydawn/orgs","repos_url":"https://api.github.com/users/polydawn/repos","events_url":"https://api.github.com/users/polydawn/events{/privacy}","received_events_url":"https://api.github.com/users/polydawn/received_events","type":"Organization","site_admin":false},"repo":{"id":31432509,"name":"repeatr","full_name":"polydawn/repeatr","owner":{"login":"polydawn","id":5307883,"avatar_url":"https://avatars.githubusercontent.com/u/5307883?v=3","gravatar_id":"","url":"https://api.github.com/users/polydawn","html_url":"https://github.com/polydawn","followers_url":"https://api.github.com/users/polydawn/followers","following_url":"https://api.github.com/users/polydawn/following{/other_user}","gists_url":"https://api.github.com/users/polydawn/gists{/gist_id}","starred_url":"https://api.github.com/users/polydawn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/polydawn/subscriptions","organizations_url":"https://api.github.com/users/polydawn/orgs","repos_url":"https://api.github.com/users/polydawn/repos","events_url":"https://api.github.com/users/polydawn/events{/privacy}","received_events_url":"https://api.github.com/users/polydawn/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/polydawn/repeatr","description":"Run it. Run it again.","fork":false,"url":"https://api.github.com/repos/polydawn/repeatr","forks_url":"https://api.github.com/repos/polydawn/repeatr/forks","keys_url":"https://api.github.com/repos/polydawn/repeatr/keys{/key_id}","collaborators_url":"https://api.github.com/repos/polydawn/repeatr/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/polydawn/repeatr/teams","hooks_url":"https://api.github.com/repos/polydawn/repeatr/hooks","issue_events_url":"https://api.github.com/repos/polydawn/repeatr/issues/events{/number}","events_url":"https://api.github.com/repos/polydawn/repeatr/events","assignees_url":"https://api.github.com/repos/polydawn/repeatr/assignees{/user}","branches_url":"https://api.github.com/repos/polydawn/repeatr/branches{/branch}","tags_url":"https://api.github.com/repos/polydawn/repeatr/tags","blobs_url":"https://api.github.com/repos/polydawn/repeatr/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/polydawn/repeatr/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/polydawn/repeatr/git/refs{/sha}","trees_url":"https://api.github.com/repos/polydawn/repeatr/git/trees{/sha}","statuses_url":"https://api.github.com/repos/polydawn/repeatr/statuses/{sha}","languages_url":"https://api.github.com/repos/polydawn/repeatr/languages","stargazers_url":"https://api.github.com/repos/polydawn/repeatr/stargazers","contributors_url":"https://api.github.com/repos/polydawn/repeatr/contributors","subscribers_url":"https://api.github.com/repos/polydawn/repeatr/subscribers","subscription_url":"https://api.github.com/repos/polydawn/repeatr/subscription","commits_url":"https://api.github.com/repos/polydawn/repeatr/commits{/sha}","git_commits_url":"https://api.github.com/repos/polydawn/repeatr/git/commits{/sha}","comments_url":"https://api.github.com/repos/polydawn/repeatr/comments{/number}","issue_comment_url":"https://api.github.com/repos/polydawn/repeatr/issues/comments{/number}","contents_url":"https://api.github.com/repos/polydawn/repeatr/contents/{+path}","compare_url":"https://api.github.com/repos/polydawn/repeatr/compare/{base}...{head}","merges_url":"https://api.github.com/repos/polydawn/repeatr/merges","archive_url":"https://api.github.com/repos/polydawn/repeatr/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/polydawn/repeatr/downloads","issues_url":"https://api.github.com/repos/polydawn/repeatr/issues{/number}","pulls_url":"https://api.github.com/repos/polydawn/repeatr/pulls{/number}","milestones_url":"https://api.github.com/repos/polydawn/repeatr/milestones{/number}","notifications_url":"https://api.github.com/repos/polydawn/repeatr/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/polydawn/repeatr/labels{/name}","releases_url":"https://api.github.com/repos/polydawn/repeatr/releases{/id}","created_at":"2015-02-27T18:12:08Z","updated_at":"2015-05-05T03:05:10Z","pushed_at":"2015-05-24T15:36:07Z","git_url":"git://github.com/polydawn/repeatr.git","ssh_url":"git@github.com:polydawn/repeatr.git","clone_url":"https://github.com/polydawn/repeatr.git","svn_url":"https://github.com/polydawn/repeatr","homepage":null,"size":2451,"stargazers_count":3,"watchers_count":3,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":4,"forks":1,"open_issues":4,"watchers":3,"default_branch":"master"}},"base":{"label":"polydawn:master","ref":"master","sha":"1b0e3ee4641e2f8b97c81a9ae057ac9515e02eb7","user":{"login":"polydawn","id":5307883,"avatar_url":"https://avatars.githubusercontent.com/u/5307883?v=3","gravatar_id":"","url":"https://api.github.com/users/polydawn","html_url":"https://github.com/polydawn","followers_url":"https://api.github.com/users/polydawn/followers","following_url":"https://api.github.com/users/polydawn/following{/other_user}","gists_url":"https://api.github.com/users/polydawn/gists{/gist_id}","starred_url":"https://api.github.com/users/polydawn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/polydawn/subscriptions","organizations_url":"https://api.github.com/users/polydawn/orgs","repos_url":"https://api.github.com/users/polydawn/repos","events_url":"https://api.github.com/users/polydawn/events{/privacy}","received_events_url":"https://api.github.com/users/polydawn/received_events","type":"Organization","site_admin":false},"repo":{"id":31432509,"name":"repeatr","full_name":"polydawn/repeatr","owner":{"login":"polydawn","id":5307883,"avatar_url":"https://avatars.githubusercontent.com/u/5307883?v=3","gravatar_id":"","url":"https://api.github.com/users/polydawn","html_url":"https://github.com/polydawn","followers_url":"https://api.github.com/users/polydawn/followers","following_url":"https://api.github.com/users/polydawn/following{/other_user}","gists_url":"https://api.github.com/users/polydawn/gists{/gist_id}","starred_url":"https://api.github.com/users/polydawn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/polydawn/subscriptions","organizations_url":"https://api.github.com/users/polydawn/orgs","repos_url":"https://api.github.com/users/polydawn/repos","events_url":"https://api.github.com/users/polydawn/events{/privacy}","received_events_url":"https://api.github.com/users/polydawn/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/polydawn/repeatr","description":"Run it. Run it again.","fork":false,"url":"https://api.github.com/repos/polydawn/repeatr","forks_url":"https://api.github.com/repos/polydawn/repeatr/forks","keys_url":"https://api.github.com/repos/polydawn/repeatr/keys{/key_id}","collaborators_url":"https://api.github.com/repos/polydawn/repeatr/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/polydawn/repeatr/teams","hooks_url":"https://api.github.com/repos/polydawn/repeatr/hooks","issue_events_url":"https://api.github.com/repos/polydawn/repeatr/issues/events{/number}","events_url":"https://api.github.com/repos/polydawn/repeatr/events","assignees_url":"https://api.github.com/repos/polydawn/repeatr/assignees{/user}","branches_url":"https://api.github.com/repos/polydawn/repeatr/branches{/branch}","tags_url":"https://api.github.com/repos/polydawn/repeatr/tags","blobs_url":"https://api.github.com/repos/polydawn/repeatr/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/polydawn/repeatr/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/polydawn/repeatr/git/refs{/sha}","trees_url":"https://api.github.com/repos/polydawn/repeatr/git/trees{/sha}","statuses_url":"https://api.github.com/repos/polydawn/repeatr/statuses/{sha}","languages_url":"https://api.github.com/repos/polydawn/repeatr/languages","stargazers_url":"https://api.github.com/repos/polydawn/repeatr/stargazers","contributors_url":"https://api.github.com/repos/polydawn/repeatr/contributors","subscribers_url":"https://api.github.com/repos/polydawn/repeatr/subscribers","subscription_url":"https://api.github.com/repos/polydawn/repeatr/subscription","commits_url":"https://api.github.com/repos/polydawn/repeatr/commits{/sha}","git_commits_url":"https://api.github.com/repos/polydawn/repeatr/git/commits{/sha}","comments_url":"https://api.github.com/repos/polydawn/repeatr/comments{/number}","issue_comment_url":"https://api.github.com/repos/polydawn/repeatr/issues/comments{/number}","contents_url":"https://api.github.com/repos/polydawn/repeatr/contents/{+path}","compare_url":"https://api.github.com/repos/polydawn/repeatr/compare/{base}...{head}","merges_url":"https://api.github.com/repos/polydawn/repeatr/merges","archive_url":"https://api.github.com/repos/polydawn/repeatr/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/polydawn/repeatr/downloads","issues_url":"https://api.github.com/repos/polydawn/repeatr/issues{/number}","pulls_url":"https://api.github.com/repos/polydawn/repeatr/pulls{/number}","milestones_url":"https://api.github.com/repos/polydawn/repeatr/milestones{/number}","notifications_url":"https://api.github.com/repos/polydawn/repeatr/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/polydawn/repeatr/labels{/name}","releases_url":"https://api.github.com/repos/polydawn/repeatr/releases{/id}","created_at":"2015-02-27T18:12:08Z","updated_at":"2015-05-05T03:05:10Z","pushed_at":"2015-05-24T15:36:07Z","git_url":"git://github.com/polydawn/repeatr.git","ssh_url":"git@github.com:polydawn/repeatr.git","clone_url":"https://github.com/polydawn/repeatr.git","svn_url":"https://github.com/polydawn/repeatr","homepage":null,"size":2451,"stargazers_count":3,"watchers_count":3,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":4,"forks":1,"open_issues":4,"watchers":3,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/polydawn/repeatr/pulls/52"},"html":{"href":"https://github.com/polydawn/repeatr/pull/52"},"issue":{"href":"https://api.github.com/repos/polydawn/repeatr/issues/52"},"comments":{"href":"https://api.github.com/repos/polydawn/repeatr/issues/52/comments"},"review_comments":{"href":"https://api.github.com/repos/polydawn/repeatr/pulls/52/comments"},"review_comment":{"href":"https://api.github.com/repos/polydawn/repeatr/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/polydawn/repeatr/pulls/52/commits"},"statuses":{"href":"https://api.github.com/repos/polydawn/repeatr/statuses/1caf5f1cef2c3734d933b24abe8d5945370c8a0b"}},"merged":true,"mergeable":null,"mergeable_state":"unknown","merged_by":{"login":"kofalt","id":265071,"avatar_url":"https://avatars.githubusercontent.com/u/265071?v=3","gravatar_id":"","url":"https://api.github.com/users/kofalt","html_url":"https://github.com/kofalt","followers_url":"https://api.github.com/users/kofalt/followers","following_url":"https://api.github.com/users/kofalt/following{/other_user}","gists_url":"https://api.github.com/users/kofalt/gists{/gist_id}","starred_url":"https://api.github.com/users/kofalt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kofalt/subscriptions","organizations_url":"https://api.github.com/users/kofalt/orgs","repos_url":"https://api.github.com/users/kofalt/repos","events_url":"https://api.github.com/users/kofalt/events{/privacy}","received_events_url":"https://api.github.com/users/kofalt/received_events","type":"User","site_admin":false},"comments":1,"review_comments":1,"commits":57,"additions":1687,"deletions":69,"changed_files":27}}