{"action":"created","issue":{"url":"https://api.github.com/repos/18F/doi-extractives-data/issues/597","labels_url":"https://api.github.com/repos/18F/doi-extractives-data/issues/597/labels{/name}","comments_url":"https://api.github.com/repos/18F/doi-extractives-data/issues/597/comments","events_url":"https://api.github.com/repos/18F/doi-extractives-data/issues/597/events","html_url":"https://github.com/18F/doi-extractives-data/issues/597","id":106886815,"number":597,"title":"Data files should match URL structure","user":{"login":"shawnbot","id":113896,"avatar_url":"https://avatars.githubusercontent.com/u/113896?v=3","gravatar_id":"","url":"https://api.github.com/users/shawnbot","html_url":"https://github.com/shawnbot","followers_url":"https://api.github.com/users/shawnbot/followers","following_url":"https://api.github.com/users/shawnbot/following{/other_user}","gists_url":"https://api.github.com/users/shawnbot/gists{/gist_id}","starred_url":"https://api.github.com/users/shawnbot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shawnbot/subscriptions","organizations_url":"https://api.github.com/users/shawnbot/orgs","repos_url":"https://api.github.com/users/shawnbot/repos","events_url":"https://api.github.com/users/shawnbot/events{/privacy}","received_events_url":"https://api.github.com/users/shawnbot/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/18F/doi-extractives-data/labels/workflow:in-progress","name":"workflow:in-progress","color":"5319e7"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/18F/doi-extractives-data/milestones/24","html_url":"https://github.com/18F/doi-extractives-data/milestones/Sprint-RedFox","labels_url":"https://api.github.com/repos/18F/doi-extractives-data/milestones/24/labels","id":1335636,"number":24,"title":"Sprint-RedFox","description":"","creator":{"login":"meiqimichelle","id":4827522,"avatar_url":"https://avatars.githubusercontent.com/u/4827522?v=3","gravatar_id":"","url":"https://api.github.com/users/meiqimichelle","html_url":"https://github.com/meiqimichelle","followers_url":"https://api.github.com/users/meiqimichelle/followers","following_url":"https://api.github.com/users/meiqimichelle/following{/other_user}","gists_url":"https://api.github.com/users/meiqimichelle/gists{/gist_id}","starred_url":"https://api.github.com/users/meiqimichelle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/meiqimichelle/subscriptions","organizations_url":"https://api.github.com/users/meiqimichelle/orgs","repos_url":"https://api.github.com/users/meiqimichelle/repos","events_url":"https://api.github.com/users/meiqimichelle/events{/privacy}","received_events_url":"https://api.github.com/users/meiqimichelle/received_events","type":"User","site_admin":false},"open_issues":8,"closed_issues":0,"state":"open","created_at":"2015-10-02T21:20:43Z","updated_at":"2015-10-07T00:01:06Z","due_on":"2015-10-09T07:00:00Z","closed_at":null},"comments":1,"created_at":"2015-09-16T23:50:11Z","updated_at":"2015-10-07T20:53:53Z","closed_at":null,"body":"In order to support the map on the resource pages, the data URLs should be made to mimic the URL structure so that we don't have to keep maintaining the rapidly growing [mess of conditionals](https://github.com/18F/doi-extractives-data/blob/75716fa1af5beb7a30a7fcb004efea61af96001a/views/resource.html#L202-L231) in `view/resource.html`. For instance, this URL:\r\n\r\n```\r\n/resource/all/revenue/US\r\n```\r\n\r\ndisplays a state and offshore (\"regional\") choropleth of total revenues across all resources, which is fed by `/data/regional/revenues.tsv` and summed up at runtime. Ignoring for now whether we should have a truly \"national\" view that summed up all of the states and just showed a line chart instead of a map, we should at least generate distinct data files for each of these views ahead of time, which will simplify the page (and JavaScript) logic and make any issues easier to debug (rather than hunting down whether we're rolling up by the right column on one page, we can just look at the generated data file).\r\n\r\nThis is something that we would probably do as part of #591, because that would allow us to ensure that one data file gets generated per URL in whatever process generates the page structure. The view-specific data files could even sit alongside the pages, e.g.\r\n\r\n```\r\nresource/\r\n  all/\r\n    revenue/\r\n      US/\r\n        index.html\r\n        data.csv\r\n\r\n      offshore/\r\n        CHU/\r\n          index.html\r\n          data.csv\r\n        ...\r\n\r\n      onshore/\r\n        AL/\r\n          index.html\r\n          data.csv\r\n        ...\r\n```\r\n\r\nMaybe we'd need a more descriptive name for our view-specific data than `data.csv`, but I think you can see where I'm going here. Then the page templates for each different level of geographical hierarchy could change which map gets displayed (using our `<eiti-map>` custom element), but they would always be colored with `data.csv` (or whatever it's called) relative to the current URL.\r\n\r\nOf course, doing this in Jekyll is going to produce a large number of files and might make build times totally unreasonable. What I don't know (and what we'll soon discover if we go down the road of #591) is whether Jekyll's incremental builds are smart enough to avoid building the entire site when, say, a single data file changes. We'll see!"},"comment":{"url":"https://api.github.com/repos/18F/doi-extractives-data/issues/comments/146324795","html_url":"https://github.com/18F/doi-extractives-data/issues/597#issuecomment-146324795","issue_url":"https://api.github.com/repos/18F/doi-extractives-data/issues/597","id":146324795,"user":{"login":"shawnbot","id":113896,"avatar_url":"https://avatars.githubusercontent.com/u/113896?v=3","gravatar_id":"","url":"https://api.github.com/users/shawnbot","html_url":"https://github.com/shawnbot","followers_url":"https://api.github.com/users/shawnbot/followers","following_url":"https://api.github.com/users/shawnbot/following{/other_user}","gists_url":"https://api.github.com/users/shawnbot/gists{/gist_id}","starred_url":"https://api.github.com/users/shawnbot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shawnbot/subscriptions","organizations_url":"https://api.github.com/users/shawnbot/orgs","repos_url":"https://api.github.com/users/shawnbot/repos","events_url":"https://api.github.com/users/shawnbot/events{/privacy}","received_events_url":"https://api.github.com/users/shawnbot/received_events","type":"User","site_admin":false},"created_at":"2015-10-07T20:53:53Z","updated_at":"2015-10-07T20:53:53Z","body":"OK, I'm working on this now to unblock #606. I'm going to match the structure of the resource page's hash URLs and place each of the relevant files in a `/data/resources/` directory:\r\n\r\n```\r\n/data/resources/:resource/:datatype/:regiontype.csv (US)\r\n/data/resources/:resource/:datatype/onshore/:region.csv (state)\r\n/data/resources/:resource/:datatype/onshore/:region/:subregion.csv (county)\r\n/data/resources/:resource/:datatype/offshore/:region.csv (offshore area)\r\n```\r\n\r\ne.g.\r\n\r\n```\r\n/data/resources/all/revenue/US.csv\r\n/data/resources/coal/revenue/onshore/CA.csv\r\n/data/resources/coal/revenue/onshore/CA/Inyo.csv\r\n/data/resources/oilgas/exports/offshore/GOA.csv\r\n```"}}