{"action":"created","issue":{"url":"https://api.github.com/repos/golang/go/issues/13517","labels_url":"https://api.github.com/repos/golang/go/issues/13517/labels{/name}","comments_url":"https://api.github.com/repos/golang/go/issues/13517/comments","events_url":"https://api.github.com/repos/golang/go/issues/13517/events","html_url":"https://github.com/golang/go/issues/13517","id":120678169,"number":13517,"title":"proposal: Vendor specification and experimental repository fetch code","user":{"login":"kardianos","id":755121,"avatar_url":"https://avatars.githubusercontent.com/u/755121?v=3","gravatar_id":"","url":"https://api.github.com/users/kardianos","html_url":"https://github.com/kardianos","followers_url":"https://api.github.com/users/kardianos/followers","following_url":"https://api.github.com/users/kardianos/following{/other_user}","gists_url":"https://api.github.com/users/kardianos/gists{/gist_id}","starred_url":"https://api.github.com/users/kardianos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kardianos/subscriptions","organizations_url":"https://api.github.com/users/kardianos/orgs","repos_url":"https://api.github.com/users/kardianos/repos","events_url":"https://api.github.com/users/kardianos/events{/privacy}","received_events_url":"https://api.github.com/users/kardianos/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/golang/go/labels/Proposal","name":"Proposal","color":"ededed"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/golang/go/milestones/30","html_url":"https://github.com/golang/go/milestones/Proposal","labels_url":"https://api.github.com/repos/golang/go/milestones/30/labels","id":1373555,"number":30,"title":"Proposal","description":"Proposals that are pending (not yet accepted/rejected).\r\n","creator":{"login":"rsc","id":104030,"avatar_url":"https://avatars.githubusercontent.com/u/104030?v=3","gravatar_id":"","url":"https://api.github.com/users/rsc","html_url":"https://github.com/rsc","followers_url":"https://api.github.com/users/rsc/followers","following_url":"https://api.github.com/users/rsc/following{/other_user}","gists_url":"https://api.github.com/users/rsc/gists{/gist_id}","starred_url":"https://api.github.com/users/rsc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rsc/subscriptions","organizations_url":"https://api.github.com/users/rsc/orgs","repos_url":"https://api.github.com/users/rsc/repos","events_url":"https://api.github.com/users/rsc/events{/privacy}","received_events_url":"https://api.github.com/users/rsc/received_events","type":"User","site_admin":false},"open_issues":32,"closed_issues":14,"state":"open","created_at":"2015-10-24T01:40:49Z","updated_at":"2015-12-28T19:24:23Z","due_on":null,"closed_at":null},"comments":29,"created_at":"2015-12-07T00:47:46Z","updated_at":"2015-12-30T03:30:04Z","closed_at":null,"body":"# Proposal: Vendor specification and experimental repository fetch code\r\n\r\nAuthor(s): Daniel Theophanes\r\n\r\nLast updated: 2015-12-06\r\n\r\n## Abstract\r\n\r\nEstablish a specification file format that lists dependency revisions and\r\na package in the golang.org/x/exp repository that discovers, reads, and downloads\r\npackages at a given revision. Tools may continue to use other formats to generate\r\nthis file.\r\n\r\n## Background\r\n\r\nMany developers wish to specify revisions of vendor dependencies without copying\r\nthem into the repository. For a case study I will bring up two:\r\n\r\n A) https://github.com/cockroachdb/cockroach\r\n\r\n B) https://github.com/gluster/glusterd2\r\n\r\n(A) uses `github.com/robfig/glock` which specifies revisions for each remote repository\r\nin file in the project root called \"GLOCKFILE\". A partial list of the file is:\r\n```\r\ncmd golang.org/x/tools/cmd/stress\r\ncmd golang.org/x/tools/cmd/stringer\r\ngithub.com/agtorre/gocolorize f42b554bf7f006936130c9bb4f971afd2d87f671\r\ngithub.com/biogo/store 3b4c041f52c224ee4a44f5c8b150d003a40643a0\r\ngithub.com/cockroachdb/c-rocksdb bf15ead80bdc205a19b3d33415b23c156a3cf371\r\ngithub.com/cockroachdb/c-snappy 5c6d0932e0adaffce4bfca7bdf2ac37f79952ccf\r\ngithub.com/cockroachdb/yacc 443154b1852a8702b07d675da6cd97cd9177a316\r\ngithub.com/coreos/etcd a423a55b142c2b9a82811604204cddbccd0a9cf9\r\n```\r\n\r\n(B) uses `github.com/Masterminds/glide` which specifies revisions for each remote\r\nrepository in a file in the project root called \"glide.yaml\". This file contains:\r\n```\r\nparent: null\r\npackage: github.com/gluster/glusterd2\r\nimport:\r\n- package: github.com/gorilla/context\r\n  version: 1c83b3eabd45b6d76072b66b746c20815fb2872d\r\n- package: gopkg.in/tylerb/graceful.v1\r\n  version: 48afeb21e2fcbcff0f30bd5ad6b97747b0fae38e\r\n- package: github.com/pborman/uuid\r\n  version: cccd189d45f7ac3368a0d127efb7f4d08ae0b655\r\n- package: github.com/gorilla/mux\r\n  version: ad4d7a5882b961e07e2626045eb995c022ac6664\r\n- package: golang.org/x/net\r\n  version: b4e17d61b15679caf2335da776c614169a1b4643\r\n- package: github.com/docker/libkv\r\n  version: 93099f38de7421e6979983652730a81e2bafd578\r\n- package: github.com/codegangsta/negroni\r\n  version: c7477ad8e330bef55bf1ebe300cf8aa67c492d1b\r\n- package: golang.org/x/sys\r\n  subpackages:\r\n  - /unix\r\n- package: github.com/meatballhat/negroni-logrus\r\n  version: dd89490b0057cca7fe3fa3885f82935dfd430c2e\r\n- package: github.com/Sirupsen/logrus\r\n  version: v0.8.7\r\n- package: github.com/hashicorp/consul\r\n  version: v0.5.2\r\n```\r\n\r\nI would like to point out a few features these tools provide:\r\n * Specify commands to fetch.\r\n * Specify repositories at a given revision.\r\n * Specify repositories at a given version.\r\n * Specify a sub-tree of packages in a given repository.\r\n\r\nRight now each vendor tool specifies these same properties in different formats.\r\nA common tool cannot be built that reads a single file and downloads the needed\r\ndependencies. This isn't a huge burden on a dedicated developer, but for a\r\nuser passing by who just wants to build the source quickly, it is an impediment.\r\n\r\n## Proposal\r\n\r\nI propose specifying a single file format that will describe packages sourced\r\noutside the project repository. I also propose adding a packge to the\r\ngolang.org/x/exp repository that discovers, reads, and optionally downloads\r\nthird party packages.\r\n\r\nFurthermore I propose using the specification found at\r\nhttps://github.com/kardianos/vendor-spec with one addition as the basis for this\r\nspecification. The addition is:\r\n```\r\nPackage []struct {\r\n\t...\r\n\t\r\n\t// Tree indicates that the specified folder, along with all sub-folders\r\n\t// are required.\r\n\tTree bool `json:\"tree\"`\r\n\r\n\t...\r\n}\r\n```\r\n\r\nBoth the specification and the proposed package will be considered experimental\r\nand subject to change or retraction until at least go1.7. This process will be\r\ndone with an eye to possibly adding this feature to `go get`.\r\n\r\n## Rationale\r\n\r\nThe vendor file format needs to be able to be read and written with standard\r\ngo packages. This adds to the possibly that `go get` could fetch packages\r\nautomatically.\r\n\r\nVendor tools exist today that download packages from a specification. They are just\r\nincompatible with each other despite using the same information to download the\r\ndependencies. If we can agree on a single format for tools to write to, even if\r\nit isn't the primary format for that tool, all tools and possibly `go get` can\r\ndownload dependencies.\r\n\r\nExisting vendor tools and their formats don't always handle corner cases or\r\ndifferent approaches. For example current tool file formats can't handle the\r\ncase of vendoring a patched version of a standard library package (this\r\nwould have been useful for `crypto/tls` forks for detecting the heartbleed\r\nattack and for accessing MS Azure).\r\n\r\nI am proposing a file format that \"govendor\" uses. I'm not trying to put my own\r\ntool as central. Infact, \"govendor\" was built to validate the \"vendor-spec\"\r\nproposal. The \"vendor-spec\" has received significant external contributions\r\nand as such \"govendor\" has changed to match the spec (and will continue to do so).\r\n\r\n## Compatibility\r\n\r\nThis will be standardization of existing practices. There is no go1 compatibility\r\nissues. Existing tools can treat the specification as a write only file.\r\n\r\n## Implementation\r\n\r\nA file format to describe vendor packages should be accepted when this\r\nproposal is accepted. Should this proposal be accepted a new package\r\nshould be added to the \"golang.org/x/exp\" repository to support reading\r\nthe vendor file and downloading packages. The author of this proposal\r\noffers to create or assist in creating this package. This would be created\r\nwithin 2 months of the proposal being accepted.\r\n\r\n## Risks\r\n\r\nIt would be ideal if other vendor tool package authors could agree to at least\r\nwrite to a standard file format informally and collaboratively. Indeed the largest\r\nrisk is if vendor tools fail to write the common file format. However I think\r\nunless there is a tangible benefit (such as `go get` support) there will continue\r\nto not be a reason to collaborate on a standard.\r\n\r\n## Open issues\r\n\r\nThe proposed standard file format uses JSON, which might be better then XML, but\r\nharder to write by then something like TOML. Tools that want the vendor file\r\nto be hand created will be forced to generate this file from a different file.\r\n\r\nThe file format specifies packages, not repositories. Repositories can be specified\r\nby using the root path to the repository and specifying `\"tree\": true`, but it isn't\r\nthe default for the format. Some people may take issue with that as they are used\r\nto or desire tools that only work at the repository level. This could be a point\r\nof division. From experience I absolutely love vendoring at the package level\r\n(this is what github.com/kardianos/govendor does by default).\r\n"},"comment":{"url":"https://api.github.com/repos/golang/go/issues/comments/167930601","html_url":"https://github.com/golang/go/issues/13517#issuecomment-167930601","issue_url":"https://api.github.com/repos/golang/go/issues/13517","id":167930601,"user":{"login":"sdboyer","id":21599,"avatar_url":"https://avatars.githubusercontent.com/u/21599?v=3","gravatar_id":"","url":"https://api.github.com/users/sdboyer","html_url":"https://github.com/sdboyer","followers_url":"https://api.github.com/users/sdboyer/followers","following_url":"https://api.github.com/users/sdboyer/following{/other_user}","gists_url":"https://api.github.com/users/sdboyer/gists{/gist_id}","starred_url":"https://api.github.com/users/sdboyer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sdboyer/subscriptions","organizations_url":"https://api.github.com/users/sdboyer/orgs","repos_url":"https://api.github.com/users/sdboyer/repos","events_url":"https://api.github.com/users/sdboyer/events{/privacy}","received_events_url":"https://api.github.com/users/sdboyer/received_events","type":"User","site_admin":false},"created_at":"2015-12-30T03:30:04Z","updated_at":"2015-12-30T03:30:04Z","body":"@ardan-bkennedy could you please provide an example of a \"kit type repo\"?"}}