{"action":"created","issue":{"url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/247","labels_url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/247/labels{/name}","comments_url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/247/comments","events_url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/247/events","html_url":"https://github.com/CausalityLtd/ponyc/issues/247","id":91310406,"number":247,"title":"Package manager","user":{"login":"kamilchm","id":1621930,"avatar_url":"https://avatars.githubusercontent.com/u/1621930?v=3","gravatar_id":"","url":"https://api.github.com/users/kamilchm","html_url":"https://github.com/kamilchm","followers_url":"https://api.github.com/users/kamilchm/followers","following_url":"https://api.github.com/users/kamilchm/following{/other_user}","gists_url":"https://api.github.com/users/kamilchm/gists{/gist_id}","starred_url":"https://api.github.com/users/kamilchm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kamilchm/subscriptions","organizations_url":"https://api.github.com/users/kamilchm/orgs","repos_url":"https://api.github.com/users/kamilchm/repos","events_url":"https://api.github.com/users/kamilchm/events{/privacy}","received_events_url":"https://api.github.com/users/kamilchm/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/CausalityLtd/ponyc/labels/question","name":"question","color":"cc317c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":17,"created_at":"2015-06-26T17:25:08Z","updated_at":"2015-07-25T15:16:10Z","closed_at":null,"body":"I saw a presentation from Big Techday 8 http://www.techcast.com/events/bigtechday8/pranner-1450/. Great one BTW ;)\r\nYou said you wish to use someones package manager and you're looking for suggestions.\r\nYou need to try http://nixos.org/nix/. You can't do it better :)\r\nSee example setup for Golang:\r\n* https://nixos.org/nixpkgs/manual/#sec-language-go\r\n* http://lethalman.blogspot.com/2015/02/developing-in-golang-with-nix-package.html"},"comment":{"url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/comments/124852942","html_url":"https://github.com/CausalityLtd/ponyc/issues/247#issuecomment-124852942","issue_url":"https://api.github.com/repos/CausalityLtd/ponyc/issues/247","id":124852942,"user":{"login":"andymcn","id":7050628,"avatar_url":"https://avatars.githubusercontent.com/u/7050628?v=3","gravatar_id":"","url":"https://api.github.com/users/andymcn","html_url":"https://github.com/andymcn","followers_url":"https://api.github.com/users/andymcn/followers","following_url":"https://api.github.com/users/andymcn/following{/other_user}","gists_url":"https://api.github.com/users/andymcn/gists{/gist_id}","starred_url":"https://api.github.com/users/andymcn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andymcn/subscriptions","organizations_url":"https://api.github.com/users/andymcn/orgs","repos_url":"https://api.github.com/users/andymcn/repos","events_url":"https://api.github.com/users/andymcn/events{/privacy}","received_events_url":"https://api.github.com/users/andymcn/received_events","type":"User","site_admin":false},"created_at":"2015-07-25T15:16:10Z","updated_at":"2015-07-25T15:16:10Z","body":"Here's my new proposal, taking into account everything that's been said recently. Please let me know if any of this is unclear or seems like a bad idea.\r\n\r\n## Proposal\r\n\r\n### Bundle specification\r\n\r\nThe specification of which bundles to import is specified in the Pony code via use commands. Bundles __have__ to be given aliases, which are then used when importing the specific packages within the bundle.\r\n\r\nBundles can be sourced remotely (remote external bundles), eg from github:\r\n\r\n```\r\nuse foo = \"github:cquinn/foolib/2.1\"\r\n```\r\n\r\nor locally (local external bundles):\r\n\r\n```\r\nuse foo = \"bundle:foolib/2.1\"\r\n```\r\n\r\nThe scope of the bundle aliased specified in these commands is not project wide. In particular, 2 different bundles do not have to use the same alias for the same third bundle.\r\n\r\nPackages to use can then be specified with other use commands:\r\n\r\n```\r\nuse \"foo/utils\"\r\n```\r\n\r\nThe \"foo\" in such a use command could only be (in order):\r\n* A previously defined bundle alias.\r\n* A local package name.\r\n* A standard library package.\r\n\r\n### Bundle versioning\r\n\r\nBoth remote and local bundles use semantic versioning. The most recent matching version is fetched. For example, if the existing versions are:\r\n\r\n* 2.3.0\r\n* 2.3.1\r\n* 2.3.2\r\n* 2.4.0\r\n* 2.4.1\r\n* 2.4.2\r\n* 2.4.3\r\n\r\nIf you ask for version 2 you'll get 2.4.3.\r\nIf you ask for 2.3 you'll get 2.3.2.\r\nIf you ask for 2.3.1 you'll get 2.3.1.\r\nIf no version is specified the latest is used (2.4.3).\r\n\r\nThe fetcher performs this version check, fetching the appropriate version.\r\n\r\nNote that a single project (or even a single source file) may require multiple versions of the same bundle.\r\n\r\n### Fetching bundles\r\n\r\nThe fetcher is invoked on the project. It walks all source files finding bundle use commands, builds a list of all required bundles and fetches them. It then does the same for the fetched bundles and keeps going until all transitively required bundles are fetched (or there's a failure).\r\n\r\nThe fetcher ignores the aliases the use commands provide for bundles, they are only used by the compiler.\r\n\r\nRemotely sourced bundles are fetched into the user's bundle store. Locally sourced external bundles could be copied into the user's bundle store, but I think it makes more sense to just use them from where-ever they are on the local system.\r\n\r\nThe list of required bundles is stored in a project wide file. This can be edited manually to specify bundle substitutions. If this file doesn't exist then the fetcher creates it.\r\n\r\nThe fetcher reads this file before fetching anything and honours any substitutions it specifies. It checks for newer versions of bundles every time it is run, but not for substituted bundles which need to be updated manually.\r\n\r\nThis file is only used in projects and is not required for external bundles and is not distributed with bundles. It is only used for specifying where to find the bundle referenced in the source code, including substitutions and versions, plus possibly for documentation and caching purposes.\r\n\r\nBundle stores will have their own files specifying what bundles they contain, including versions, checksums, etc. The format of these files and the information stored in them is entirely an implementation detail of the fetcher.\r\n\r\n### Building\r\n\r\nThe compiler processes the bundle list file and uses it to locate the bundles it needs. Any required bundles that are not present are treated as errors, the compiler never tries to fetch anything itself.\r\n\r\nThe compiler does not try to edit the bundle file.\r\n\r\nIf there are no external bundles used then the compiler doesn't need the bundle file and won't complain if it doesn't exist. If any external bundles are used then the bundle file must exist or the compiler throws an error.\r\n\r\nThis allows for the compiler to search up its directory tree to find the bundle file (in the project's root directory) regardless of where the compilation starts. But also allows simple projects (that use no external bundles) not to need the file or ever have to run the fetcher.\r\n\r\n### Invoking\r\n\r\nThere is a single top level tool, called pony, that the user calls with various arguments to perform all operations. This tool will invoke the fetcher and compiler (and possibly others) that are separate executables.\r\n\r\nThere is no technical reason we couldn't have it so the compiler does all of this, but it seems reasonable to split it.\r\n\r\n### Bundle file format\r\n\r\nThe bundle file is autogenerated, but needs to be human readable and editable.\r\n\r\nIt could be a Pony file, but I don't think that's sensible. It only contains bundle information, not general code, and the bundle substitution information won't fit into existing Pony syntax.\r\n\r\nSo instead we should use an existing standard format. Json and yaml are both sensible options.\r\n\r\nHere's a sample bundle list file in pseudo format. The foolib entry is purely autogenerated. The barlib entry has been manually changed to substitute a local version of the bundle, the substitute keyword is used so the fetcher knows not to try to update the version of the bundle used. The awesomelib entry is an autogenerated entry for a local bundle.\r\n\r\n```\r\nbundle github:cquinn/foolib/2.1\r\n  store /home/andy/ponystore/github_cquinn/foolib/2.1.3/\r\nbundle github:cquinn/barlib/3\r\n  substitute /home/andy/mylibs/mybar/0.1.4/\r\nbundle bundle:awesomelib\r\n  store /home/andy/mylibs/awesomelib/0.0.0/\r\n```\r\n\r\nWe could also store other project related information in the bundle list file. For example, this would stop us having to specify the output directory on the command line every time the compiler was invoked. I'm not sure whether this is a good idea or not.\r\n\r\n### Things I've skipped over\r\n\r\n* The fetcher needs to know where to look for local bundles and where to copy remote bundles to. I'm assuming this will be done with some combination of environment variables, command line options and standard relative paths, much as the compiler currently does things. The compiler won't need any such mechanism because the bundle file will tell it where to find bundles.\r\n\r\n* The scope of the bundle aliases created by use commands could be file, package or bundle wide. However, since this doesn't affect the fetcher and is purely a compiler issue I'm not considering it important for this discussion.\r\n\r\n* I've completely ignored the standard library. At the moment it isn't versioned, however it really should be. If we don't version it then we won't be able to change it later without annoying existing users. It could be handled just as a special case of a local bundle, with an implicit use command. Or we might want to do something different.\r\n\r\n* I haven't mentioned the question I previously raised of having multiple local bundle stores, possibly some being read-only. I don't think this is actually very useful for I'm ignoring it for now. However, nothing in this proposal should prevent it being added later.\r\n\r\n* There is no mechanism here for substituting one bundle for another in only part of a project. Again I'm not sure whether that's actually a useful ability. Also it is possible to achieve such an affect by editing all the relevant use commands in the source code.\r\n"}}