{"action":"created","issue":{"url":"https://api.github.com/repos/pelias/openstreetmap/issues/45","labels_url":"https://api.github.com/repos/pelias/openstreetmap/issues/45/labels{/name}","comments_url":"https://api.github.com/repos/pelias/openstreetmap/issues/45/comments","events_url":"https://api.github.com/repos/pelias/openstreetmap/issues/45/events","html_url":"https://github.com/pelias/openstreetmap/pull/45","id":62971534,"number":45,"title":"improve osm name tag parsing","user":{"login":"missinglink","id":738069,"avatar_url":"https://avatars.githubusercontent.com/u/738069?v=3","gravatar_id":"","url":"https://api.github.com/users/missinglink","html_url":"https://github.com/missinglink","followers_url":"https://api.github.com/users/missinglink/followers","following_url":"https://api.github.com/users/missinglink/following{/other_user}","gists_url":"https://api.github.com/users/missinglink/gists{/gist_id}","starred_url":"https://api.github.com/users/missinglink/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/missinglink/subscriptions","organizations_url":"https://api.github.com/users/missinglink/orgs","repos_url":"https://api.github.com/users/missinglink/repos","events_url":"https://api.github.com/users/missinglink/events{/privacy}","received_events_url":"https://api.github.com/users/missinglink/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pelias/openstreetmap/labels/in%20review","name":"in review","color":"ededed"}],"state":"open","locked":false,"assignee":{"login":"missinglink","id":738069,"avatar_url":"https://avatars.githubusercontent.com/u/738069?v=3","gravatar_id":"","url":"https://api.github.com/users/missinglink","html_url":"https://github.com/missinglink","followers_url":"https://api.github.com/users/missinglink/followers","following_url":"https://api.github.com/users/missinglink/following{/other_user}","gists_url":"https://api.github.com/users/missinglink/gists{/gist_id}","starred_url":"https://api.github.com/users/missinglink/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/missinglink/subscriptions","organizations_url":"https://api.github.com/users/missinglink/orgs","repos_url":"https://api.github.com/users/missinglink/repos","events_url":"https://api.github.com/users/missinglink/events{/privacy}","received_events_url":"https://api.github.com/users/missinglink/received_events","type":"User","site_admin":false},"milestone":null,"comments":1,"created_at":"2015-03-19T12:26:48Z","updated_at":"2015-03-19T16:09:59Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/pelias/openstreetmap/pulls/45","html_url":"https://github.com/pelias/openstreetmap/pull/45","diff_url":"https://github.com/pelias/openstreetmap/pull/45.diff","patch_url":"https://github.com/pelias/openstreetmap/pull/45.patch"},"body":"This PR introduces far more strict name parsing to avoid errors and reduce the amount of data being stored on disk.\n\nPrior to this PR we have been very liberal with the `name` tags we import from OSM, it is convenient to import as many names as possible but leads to odd errors such as [1]  - (caused by a tag with an extraneous colon and no suffix:)\n\n```bash\n{ 'name:': 'foo' }\n       ^\n```\n\nI did some research in to the tags in osm which contain 'name:' and found there are a huge variety of tags [3] well outside the suffix values mentioned in the wiki [2][4], I've committed the list and a script to repeat that in the `./test/fixtures` directory.\n\nIn order to avoid things like ```'name:', 'name:*', 'name:ဗမာ', 'name:<阪神電鉄>'``` ending up in the index I chose to adopt a strict whitelist approach, so if the suffix doesn't appear in the array of values specified in `./config/localized_keys.js` [5] it will be rejected.\n\nThe rest of the PR is adding more tests to prevent regressions.\n\n[1] https://github.com/pelias/openstreetmap/issues/44\n[2] http://wiki.openstreetmap.org/wiki/Namespace#Language_code_suffix\n[3] https://github.com/pelias/openstreetmap/blob/name_parsing/test/fixtures/name-tags.json\n[4] https://github.com/pelias/openstreetmap/blob/name_parsing/test/fixtures/official-name-tags.json\n[5] https://github.com/pelias/openstreetmap/blob/name_parsing/config/localized_keys.js\n\nFixes #44"},"comment":{"url":"https://api.github.com/repos/pelias/openstreetmap/issues/comments/83645668","html_url":"https://github.com/pelias/openstreetmap/pull/45#issuecomment-83645668","issue_url":"https://api.github.com/repos/pelias/openstreetmap/issues/45","id":83645668,"user":{"login":"dianashk","id":5446511,"avatar_url":"https://avatars.githubusercontent.com/u/5446511?v=3","gravatar_id":"","url":"https://api.github.com/users/dianashk","html_url":"https://github.com/dianashk","followers_url":"https://api.github.com/users/dianashk/followers","following_url":"https://api.github.com/users/dianashk/following{/other_user}","gists_url":"https://api.github.com/users/dianashk/gists{/gist_id}","starred_url":"https://api.github.com/users/dianashk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dianashk/subscriptions","organizations_url":"https://api.github.com/users/dianashk/orgs","repos_url":"https://api.github.com/users/dianashk/repos","events_url":"https://api.github.com/users/dianashk/events{/privacy}","received_events_url":"https://api.github.com/users/dianashk/received_events","type":"User","site_admin":false},"created_at":"2015-03-19T16:09:59Z","updated_at":"2015-03-19T16:09:59Z","body":"This looks great. Great idea using taginfo to validate tag names. I'd love to have this functionality in Fences (admin boundaries project), so maybe it's something we can extract into a standalone util in the near future. (I volunteer!)\r\n\r\nShould add taginfo and async to devDependencies in order for fetch-taginfo.js to work.\r\nIt would also be nice to have a script that automagically updates the `localized_keys` file. So you could do `npm run update-name-keys`. My understanding is that currently you'd run `fetch-taginfo` manually and copy the tag keys with `name:` prefix to your file by hand.\r\n\r\nConsider renaming `localized_keys.js` to something more coupled with `name` (`localized_name_keys.js`?)."}}