{"action":"created","issue":{"url":"https://api.github.com/repos/w3c/csvw/issues/702","labels_url":"https://api.github.com/repos/w3c/csvw/issues/702/labels{/name}","comments_url":"https://api.github.com/repos/w3c/csvw/issues/702/comments","events_url":"https://api.github.com/repos/w3c/csvw/issues/702/events","html_url":"https://github.com/w3c/csvw/issues/702","id":103462586,"number":702,"title":"Various improvements including greater alignment with Tabular Data Package and JSON Table Schema","user":{"login":"rgrp","id":180658,"avatar_url":"https://avatars.githubusercontent.com/u/180658?v=3","gravatar_id":"","url":"https://api.github.com/users/rgrp","html_url":"https://github.com/rgrp","followers_url":"https://api.github.com/users/rgrp/followers","following_url":"https://api.github.com/users/rgrp/following{/other_user}","gists_url":"https://api.github.com/users/rgrp/gists{/gist_id}","starred_url":"https://api.github.com/users/rgrp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rgrp/subscriptions","organizations_url":"https://api.github.com/users/rgrp/orgs","repos_url":"https://api.github.com/users/rgrp/repos","events_url":"https://api.github.com/users/rgrp/events{/privacy}","received_events_url":"https://api.github.com/users/rgrp/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/w3c/csvw/labels/For%20Proposed%20Rec","name":"For Proposed Rec","color":"006b75"},{"url":"https://api.github.com/repos/w3c/csvw/labels/Metadata%20vocabulary%20document","name":"Metadata vocabulary document","color":"5319e7"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-27T09:06:20Z","updated_at":"2015-09-23T15:11:49Z","closed_at":"2015-09-23T14:44:26Z","body":"This issue suggests a variety of improvements to the current version of the specification. It is the result of substantial reflection on the current version of the spec. It is something of an \"omnibus\" issue and it could be broken up in more bite-sized chunks.\r\n\r\nThe various improvements are both made in themselves and also to seek greater alignment where possible with the [Tabular Data Package][tdp] and [JSON Table Schema][jts] specifications.\r\n\r\nAs people know, this spec was originally heavily based on Tabular Data Package. Over time it seems to have drifted somewhat. I would like to suggest various revisions to bring closer alignment. Why do this? If we can converge these specs then:\r\n\r\n[tdp]: http://dataprotocols.org/tabular-data-package/\r\n[jts]: http://dataprotocols.org/json-table-schema/\r\n\r\n* it is possible for tooling to be easily reused between the two.\r\n* it is possible that there could be complete convergence on all or parts of the spec that would allow for direct reuse and/or merge\r\n* we reduce confusion amongst the community\r\n\r\nIn addition, i would note that Tabular Data Package has seen several years of real-world use and so reuse brings the benefits of that experience.\r\n\r\n## Suggested Changes\r\n\r\n### General Metadata\r\n\r\n* #712 - Rename `tables` attribute to `resources`\r\n* #713 - Rename `tableSchema` to simple `schema`\r\n\r\n#### Tables, Columns etc\r\n\r\n* #715 Remove `aboutUrl` and `valueUrl`\r\n* #716 `titles` -> `title` and require it to be single valued.\r\n* `separator`: rather than allow this on column require it to be part of dialect\r\n  * why specially move this up on to the column object - treat it like every other dialect property\r\n* `columns` vs `fields` (JTS): this can be resolved by an upgrade in JTS to support `columns` name\r\n* #720 - rework constraints into a dedicated property - this also makes datatype single valued again\r\n* #721 - Make datatype single valued again\r\n* #717 Alignment of datatypes between [JSON Table Schema][jts] and this schema\r\n* Remove `rowTitles`\r\n  * Dubious of need. We should always strive for parsimony.\r\n  * Minor point\r\n\r\n### Other Suggested Changes\r\n\r\n* #718 - virtual columns: remove from spec.\r\n* cf #715 - Remove explicit support for URI Templates\r\n  * The purpose of these - as I understand it - is primarily to support URI generation in `aboutUrl`. As above this is of limited value and these add substantial implementational and cognitive complexity to the specification.\r\n* #722 - Move transformation entirely out of the main spec\r\n* #722 Keep transformation and conversion hints separate from the main spec\r\n\r\n## Discussions (not specific suggestions)\r\n\r\n### Namespacing\r\n\r\nI would recommend keeping namespacing to a minimum, at least in our examples for properties (e.g. `dc:title` vs `title`).\r\n\r\nWe can, and should, definitely allow namespacing - and it comes automatically really.\r\n\r\nHowever, a key goal is simplicity and namespacing adds complexity (i have to work out what that `dc:` means ...)\r\n\r\n### i18n\r\ni18n is one of the more complex issues and presents real challenges for trading off expressiveness vs simplicity and ease of use. My sense is that the ultimate JSON-LD route is rather heavy-weight e.g.\r\n\r\n    `\"title\": {\"@value\": \"The title of this Table\", \"@language\": \"en\"}`\r\n\r\nI would suggest the earlier JSON-LD approach of appending \"@\" to fields e.g.\r\n\r\n    `\"title@en\": \"The title of this table\"`\r\n\r\nMy preference for the latter is that it will be much easier to use and read for most non-expert users. I also think it \"degrades\" more nicely in that in the \"title@lang\" approach you do not have to change the default language value when you add new languages (this value is probably the one most important to most users - they may just ignore all other languages). That said, I acknowledge that either approach has challenges (e.g. this latter one means that parsing property values must be i18n aware). \r\n\r\n### Example\r\n\r\nTo illustrate some of the changes here is the current main example and then a rewritten version:\r\n\r\n```\r\n{\r\n  \"@context\": [\"http://www.w3.org/ns/csvw\", {\"@language\": \"en\"}],\r\n  \"url\": \"tree-ops.csv\",\r\n  \"dc:title\": \"Tree Operations\",\r\n  \"dcat:keyword\": [\"tree\", \"street\", \"maintenance\"],\r\n  \"dc:publisher\": {\r\n    \"schema:name\": \"Example Municipality\",\r\n    \"schema:url\": {\"@id\": \"http://example.org\"}\r\n  },\r\n  \"dc:license\": {\"@id\": \"http://opendefinition.org/licenses/cc-by/\"},\r\n  \"dc:modified\": {\"@value\": \"2010-12-31\", \"@type\": \"xsd:date\"},\r\n  \"tableSchema\": {\r\n    \"columns\": [{\r\n      \"name\": \"GID\",\r\n      \"titles\": [\"GID\", \"Generic Identifier\"],\r\n      \"dc:description\": \"An identifier for the operation on a tree.\",\r\n      \"datatype\": \"string\",\r\n      \"required\": true\r\n    }, {\r\n      \"name\": \"on_street\",\r\n      \"titles\": \"On Street\",\r\n      \"dc:description\": \"The street that the tree is on.\",\r\n      \"datatype\": \"string\"\r\n    }, {\r\n      \"name\": \"species\",\r\n      \"titles\": \"Species\",\r\n      \"dc:description\": \"The species of the tree.\",\r\n      \"datatype\": \"string\"\r\n    }, {\r\n      \"name\": \"trim_cycle\",\r\n      \"titles\": \"Trim Cycle\",\r\n      \"dc:description\": \"The operation performed on the tree.\",\r\n      \"datatype\": \"string\"\r\n    }, {\r\n      \"name\": \"inventory_date\",\r\n      \"titles\": \"Inventory Date\",\r\n      \"dc:description\": \"The date of the operation that was performed.\",\r\n      \"datatype\": {\"base\": \"date\", \"format\": \"M/d/yyyy\"}\r\n    }],\r\n    \"primaryKey\": \"GID\",\r\n    \"aboutUrl\": \"#gid-{GID}\"\r\n  }\r\n}\r\n```\r\n\r\n**Rewritten version**\r\n\r\n```\r\n{\r\n  \"url\": \"tree-ops.csv\",\r\n  \"title\": \"Tree Operations\",\r\n  \"keywords\": [\"tree\", \"street\", \"maintenance\"],\r\n  \"publisher\": {\r\n    \"name\": \"Example Municipality\",\r\n    \"url\": \"http://example.org\"\r\n  },\r\n  \"license\": \"http://opendefinition.org/licenses/cc-by/\",\r\n  \"modified\": \"2010-12-31\",\r\n  # note I would suggest always nesting this within a resources array but that's a bigger discussion\r\n  \"schema\": {\r\n    \"columns\": [{\r\n      \"name\": \"GID\",\r\n      \"title\": \"Generic Identifier\",\r\n      \"description\": \"An identifier for the operation on a tree.\",\r\n      \"datatype\": \"string\",\r\n      \"constraints\": {\r\n         \"required\": true\r\n       }\r\n    }, {\r\n      \"name\": \"on_street\",\r\n      \"title\": \"On Street\",\r\n      \"description\": \"The street that the tree is on.\",\r\n      \"datatype\": \"string\"\r\n    }, {\r\n      \"name\": \"species\",\r\n      \"title\": \"Species\",\r\n      \"description\": \"The species of the tree.\",\r\n      \"datatype\": \"string\"\r\n    }, {\r\n      \"name\": \"trim_cycle\",\r\n      \"title\": \"Trim Cycle\",\r\n      \"description\": \"The operation performed on the tree.\",\r\n      \"datatype\": \"string\"\r\n    }, {\r\n      \"name\": \"inventory_date\",\r\n      \"title\": \"Inventory Date\",\r\n      \"description\": \"The date of the operation that was performed.\",\r\n      \"datatype\": \"date\",\r\n      \"format\": \"M/d/yyyy\"\r\n    }],\r\n    \"primaryKey\": \"GID\",\r\n  }\r\n}\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/w3c/csvw/issues/comments/142634819","html_url":"https://github.com/w3c/csvw/issues/702#issuecomment-142634819","issue_url":"https://api.github.com/repos/w3c/csvw/issues/702","id":142634819,"user":{"login":"rgrp","id":180658,"avatar_url":"https://avatars.githubusercontent.com/u/180658?v=3","gravatar_id":"","url":"https://api.github.com/users/rgrp","html_url":"https://github.com/rgrp","followers_url":"https://api.github.com/users/rgrp/followers","following_url":"https://api.github.com/users/rgrp/following{/other_user}","gists_url":"https://api.github.com/users/rgrp/gists{/gist_id}","starred_url":"https://api.github.com/users/rgrp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rgrp/subscriptions","organizations_url":"https://api.github.com/users/rgrp/orgs","repos_url":"https://api.github.com/users/rgrp/repos","events_url":"https://api.github.com/users/rgrp/events{/privacy}","received_events_url":"https://api.github.com/users/rgrp/received_events","type":"User","site_admin":false},"created_at":"2015-09-23T15:11:49Z","updated_at":"2015-09-23T15:11:49Z","body":"@iherman @JeniT  I should add a bit of context since unfortunately I could not make telco today due to prior commitments.\r\n\r\nMy understanding from conversation with Jeni last week, was the following:\r\n\r\n- If we make \"substantive\" changes we will need to \"reset\" the clock and go back into a 6m process\r\n- That it will be impossible - or at least very unlikely - to receive any extension on the WG's charter.\r\n\r\n=> Changes at this point will put the \"spec at risk\"\r\n\r\nIn my conversation with Jeni I noted that:\r\n\r\n- I think the spec could be substantially improved in several ways. Few of these involve actual removal or substantial alteration in features but more in a refactoring (and such refactoring would, in fact, take us back towards earlier versions)\r\n- At the same time, if A. (some of) these changes would trigger a \"reset\" + B. extension is not possible => we run the risk of not reporting out a spec.\r\n- Given that I would not want to jeopardize the efforts of the group, if it was truly the case that no extension was possible, then I accepted that we would have to leave these changes\r\n  - At the same time, i think our ultimate commitment should be a great spec ... (but hard choices have to be made and Jeni made a good case for pragmatism)\r\n  - I think it unfortunate we cannot secure some extension but understand the constraints that W3C operates under. I note also that more time would also provide more time for implementations to be trialled and feedback from that to be incorporated ...\r\n\r\nQuestion of support for changes:\r\n\r\n- In addition, there was a question of whether there was support for these items e.g. Jeni made good point that she wasn't sure that datatypes simplification was a good idea given desire/need to align with xsd etc\r\n- We agreed it wasn't really clear how widely supported these changes were (given that attendance at telcos at present is limited to core members). To really know we'd have to consult more heavily and actively.\r\n  - We'd also really need to prep two versions of the spec side by side (its almost impossible to compare individual issue by individual issue since many of the changes interact)\r\n- However, as Jeni pointed out the very act of seeking to consult actively to find out how many others shared these views would not necessarily be helpful (e.g. it would stir up debate at a late stage - and what happened if there was support: we'd then be back at square one etc)\r\n- I therefore understood that, at this point, we probably did not want to do the consultation to see if there was support, as that, in itself, could put the \"spec at risk\" (i.e. we would not ship on the required timeline or next February)\r\n\r\n\r\n"}}