{"action":"created","issue":{"url":"https://api.github.com/repos/whatwg/loader/issues/52","labels_url":"https://api.github.com/repos/whatwg/loader/issues/52/labels{/name}","comments_url":"https://api.github.com/repos/whatwg/loader/issues/52/comments","events_url":"https://api.github.com/repos/whatwg/loader/issues/52/events","html_url":"https://github.com/whatwg/loader/issues/52","id":82792402,"number":52,"title":"Why a locate hook is unnecessary","user":{"login":"guybedford","id":598730,"avatar_url":"https://avatars.githubusercontent.com/u/598730?v=3","gravatar_id":"","url":"https://api.github.com/users/guybedford","html_url":"https://github.com/guybedford","followers_url":"https://api.github.com/users/guybedford/followers","following_url":"https://api.github.com/users/guybedford/following{/other_user}","gists_url":"https://api.github.com/users/guybedford/gists{/gist_id}","starred_url":"https://api.github.com/users/guybedford/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guybedford/subscriptions","organizations_url":"https://api.github.com/users/guybedford/orgs","repos_url":"https://api.github.com/users/guybedford/repos","events_url":"https://api.github.com/users/guybedford/events{/privacy}","received_events_url":"https://api.github.com/users/guybedford/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-05-30T11:49:17Z","updated_at":"2015-05-30T12:56:55Z","closed_at":null,"body":"In creating the upgrade path in SystemJS for permitting URLs as module identifiers, it has turned out best to deprecate the locate hook. This may be over-explaining the obvious or dwelling on decisions already made, but coming to this conclusion has taken me a surprising amount of consideration so I'd like to describe the reasoning behind this here to retain some reference for the decision and attempt to leave somewhat reasoned feedback.\r\n\r\nThe question that started this was whether we should re-introduce the locate hook into the specification. Re-introducing the locate hook will enable normalize to normalize module names into a custom schema that can be defined by the loader implementation and form the string names that are stored in the module registry. Locate then handles the final resolution into URLs that can be fetched.\r\n\r\nThe justification for considering this was to retain compatibility with AMD-style module loading where we have a baseURL-schema. In this schema, modules names are stored in the registry always as plain names relative to some baseURL. jspm also uses its own schema in the registry to refer to modules such as `npm:module@x.y.z`.\r\n\r\nThere is a draw to having the sense of storing these universal schema names inside of the module registry as a portable naming system but I'd argue this lure is mostly one of elegance as opposed to practicality.\r\n\r\nI've implemented the baseURL-schema normalization in the current SystemJS, and have been experimenting recently with at least three different complete implementations of normalization of a custom schema alongside URLs (the new requirements of the spec, which completely make sense).\r\n\r\n_In the end, trying to make a custom schema work alongside URLs in the same registry space, ends up causing more issues, for no practical gain._\r\n\r\n#### Dot Normalization\r\n\r\nAs soon as we allow both AMD-style module IDs relative to some baseURL alongside URLs, the first issue we hit is the need to define \"dot normalization\". This basically means that relative normalization needs to be defined for the subset of both URLs and non-URLs.\r\n\r\nIt's not a lot of code, but it is the first sign here that we're duplicating work.\r\n\r\n#### Non-uniqueness\r\n\r\nThe next issue we have is the non-uniqueness of our schema. This issue here is that `import '/local/path.js'` is now distinct and separate to the module at `import 'local/path.js'` in the scenario where `baseURL='/'` (one resolves as a name and the other as a URL). This will cause confusion as we are allowing the same unique module to be referred to by two different possible names breaking a key principle of the registry being unique.\r\n\r\nHaving two ways to refer to the same module is a bug waiting to happen, causing problems for configuration (which variation do we configure?), creating the possibility of a module being executed twice, and interfering with bundling workflows.\r\n\r\nExpecting the user to know that they should write `import('x')` instead of `import('./x')` arbitrarily is a hard ask.\r\n\r\nThis leads down a road of trying to catch these uniqueness issues in the normalization pipeline itself, which then ends up becoming URL normalization, followed by a reverse normalization into the schema.\r\n\r\n#### All schemas have the non-uniques problem\r\n\r\nSchema non-uniqueness with URLs applies to any custom schema chosen that maps to URLs, not just the baseURL system. Even if we come up with the perfect custom naming schema, as soon as we want that schema to co-exist alongside URL requests we hit these issues.\r\n\r\nIn order to retain unique identification and configuration of modules, one ends up normalizing from schema space into URL space, and then reverse-normalizing back into schema space at the end of normalize, before resolving back into URLs from the schema in locate, just in order to have our perfect schema names stored in the registry.\r\n\r\nAdd to this the idea of a configuration space consisting of both schema and URL identifiers as well, and this compounds the problem even further.\r\n\r\nOne ends up swapping between spaces in such a way that URLs become the primary space anyway, and we're just pretending that the schema is the primary space.\r\n\r\n#### Beyond the baseURL-schema\r\n\r\nAnother common issue with baseURLs is that when back-tracking below the baseURL, we end up with \"normalized\" paths looking like \"../../module.js\", which is really not acceptable for a naming system either.\r\n\r\nIf we return to the question of what AMD's baseURL schema is really trying to accomplish, the core principle is one of portability of modules, which is completely in agreement with what we should be aiming for. URLs are obviously not a portable naming system for modules (modules can move between environments and hence change URL), so the question is simply how to maintain portability of modules in spite of using URLs?\r\n\r\n#### URLs are the schema\r\n\r\nIt turns out to be very simple to do this - normalization is seen as the process of converting a \"portable module name\" into an \"environment-specific name\". And the most environment-specific name is the URL which we store in the module registry.\r\n\r\nThe concept that we need to have a registry based on our perfect portable schema is flawed. We still keep our schema if we like - which we can bundle into just the same:\r\n\r\n```javascript\r\nSystem.register('custom:portable/schema', ...);\r\n```\r\n\r\nWhere the name above name is normalized into a resolved name of `http://www.site.com/packages/custom/portable/schema.js` by the loader when being processed and stored in the registry (bundle names are now treated as unnormalized).\r\n\r\nThere is no big loss that the registry now contains this value under an environment-specific URL instead of the schema. One can just accept that any lookup into the registry must pass through a normalization phase first:\r\n\r\n```javascript\r\n// lookup a module by its schema name by passing through a simple URL-normalization first\r\nReflect.loader.lookup(Reflect.loader.schemaToURL('custom:portable/schema'));\r\n```\r\n\r\nIf an implementor really wants to use a custom schema, make the schema URL-based and add the implementation to the fetch hook so everything works out well anyway:\r\n\r\n```javascript\r\nimport 'custom:///portable/schema'\r\n```\r\n\r\nThe other consequence of using URLs is that configuration then always goes through a normalization phase itself:\r\n\r\n```javascript\r\nReflect.loader.configure({\r\n  module: {\r\n    './some/local/module.js': {\r\n      moduleFormat: 'CommonJS'\r\n    }\r\n  }\r\n});\r\n```\r\n\r\nThe above would normalize the above configuration into `http://site.com/local/path/some/local/module.js`.\r\n\r\nThe benefit of this is that users don't need to understand the special naming schema - they can just reference modules as URLs exactly as they expect and correctly configure things without needing to have studied the system in detail.\r\n\r\nOne implication here for implementors is that build systems wanting to use portable naming system schemas need to reverse-map the schemas at build time from URLs in the registry, but that is a very minimal cost and a straightforward 1-1 mapping. \r\n\r\nI've yet to hear a single use case that is lost by enforcing that the registry is only to store URLs - the justifications for allowing the registry to store a custom schema seem to cling to dated models due to history, while there are many benefits as described to both implementors and users in enforcing URLs as the schema and keeping the locate hook deprecated."},"comment":{"url":"https://api.github.com/repos/whatwg/loader/issues/comments/107038401","html_url":"https://github.com/whatwg/loader/issues/52#issuecomment-107038401","issue_url":"https://api.github.com/repos/whatwg/loader/issues/52","id":107038401,"user":{"login":"guybedford","id":598730,"avatar_url":"https://avatars.githubusercontent.com/u/598730?v=3","gravatar_id":"","url":"https://api.github.com/users/guybedford","html_url":"https://github.com/guybedford","followers_url":"https://api.github.com/users/guybedford/followers","following_url":"https://api.github.com/users/guybedford/following{/other_user}","gists_url":"https://api.github.com/users/guybedford/gists{/gist_id}","starred_url":"https://api.github.com/users/guybedford/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/guybedford/subscriptions","organizations_url":"https://api.github.com/users/guybedford/orgs","repos_url":"https://api.github.com/users/guybedford/repos","events_url":"https://api.github.com/users/guybedford/events{/privacy}","received_events_url":"https://api.github.com/users/guybedford/received_events","type":"User","site_admin":false},"created_at":"2015-05-30T12:56:55Z","updated_at":"2015-05-30T12:56:55Z","body":"One caveat to this - we still allow special modules to be stored in the registry like `@empty` or `@loader`. The way we handle this in the latest SystemJS is to not URL-normalize a module if it is a plain name and already in the registry (which can be determined by a simple lookup). This way special environment-independent system modules can still be defined and used."}}