{"action":"created","issue":{"url":"https://api.github.com/repos/mapbox/mapbox-gl-style-spec/issues/249","labels_url":"https://api.github.com/repos/mapbox/mapbox-gl-style-spec/issues/249/labels{/name}","comments_url":"https://api.github.com/repos/mapbox/mapbox-gl-style-spec/issues/249/comments","events_url":"https://api.github.com/repos/mapbox/mapbox-gl-style-spec/issues/249/events","html_url":"https://github.com/mapbox/mapbox-gl-style-spec/issues/249","id":59892742,"number":249,"title":"data-driven styles","user":{"login":"jfirebaugh","id":98601,"avatar_url":"https://avatars.githubusercontent.com/u/98601?v=3","gravatar_id":"","url":"https://api.github.com/users/jfirebaugh","html_url":"https://github.com/jfirebaugh","followers_url":"https://api.github.com/users/jfirebaugh/followers","following_url":"https://api.github.com/users/jfirebaugh/following{/other_user}","gists_url":"https://api.github.com/users/jfirebaugh/gists{/gist_id}","starred_url":"https://api.github.com/users/jfirebaugh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jfirebaugh/subscriptions","organizations_url":"https://api.github.com/users/jfirebaugh/orgs","repos_url":"https://api.github.com/users/jfirebaugh/repos","events_url":"https://api.github.com/users/jfirebaugh/events{/privacy}","received_events_url":"https://api.github.com/users/jfirebaugh/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/mapbox/mapbox-gl-style-spec/labels/implementation-needed","name":"implementation-needed","color":"fbca04"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-03-05T01:51:21Z","updated_at":"2015-10-02T23:00:55Z","closed_at":null,"body":"This ticket summarizes my current thinking on data-driven styles based on @lucaswoj's prototype in https://github.com/mapbox/mapbox-gl-js/pull/1012 and https://github.com/mapbox/mapbox-gl-function/pull/1, and discussions in the following tickets:\r\n\r\n* https://github.com/mapbox/mapbox-gl-style-spec/issues/226 (Stop functions based on arbitrary properties (not zoom))\r\n* https://github.com/mapbox/mapbox-gl-style-spec/issues/105 (Support tokens in number properties)\r\n* https://github.com/mapbox/mapbox-gl-style-spec/issues/47 (Support basic math operators)\r\n* https://github.com/mapbox/mapbox-gl-js/issues/146 (support per-feature style attributes)\r\n\r\nWe're going to enable data-driven styles via an expansion and rationalization of the existing [function](https://www.mapbox.com/mapbox-gl-style-spec/#function) datatype, drawing on concepts from [D3 Scales](https://github.com/mbostock/d3/wiki/Scales) and the [declarative grammar established for them](https://github.com/trifacta/vega/wiki/Scales) by [vega](https://github.com/trifacta/vega).\r\n\r\nThis will require pretty extensive changes to the style specification and renderers.\r\n\r\n## Style spec changes\r\n\r\nGL style functions currently support only one kind of input: numeric zoom level. No other parameters or types may be used to specify the domain. Additionally, only simple piecewise linear or exponential mappings are supported.\r\n\r\nData-driven styles require the extension of functions to support:\r\n\r\n1. user-specified input domains with values taken from vector data properties\r\n2. discrete (non-numeric) input domains, such as names or categories\r\n3. other types of mappings, such as log, quantile, and threshold scales, and potentially mappings defined by arbitrary arithmetic expressions\r\n\r\nThese extensions will make GL \"functions\" almost exactly equivalent to what D3 and vega call scales, and I suggest we rename the GL concept to match. I'm going to use the term \"scale\" for the rest of this description.\r\n\r\n**1** will be implemented by adding a required `\"property\"` property to the scale declaration. The value is the name of a vector data property to be used for the input domain, or the special value `\"$zoom\"` to indicate current zoom level input (or current tile zoom, for layout properties). In the future we may consider adding additional special values such as `\"$type\"`, `\"$center\"`, `\"$bearing\"`, etc.\r\n\r\n**2** and **3** will be implemented by adding a required `\"type\"` property to the scale declaration, standardizing on the D3/vega scale types that correspond to GL's existing function types, and adding an `\"ordinal\"` scale type for the support of discrete input domains. The `\"linear\"`, `\"pow\"`, and `\"quantize\"` scale types cover the current functionality of our `\"interpolated\"` and `\"piecewise-constant\"` function types, and for them we'll adopt the implementation semantics that D3 uses. This will probably result in some slight differences in things like how out-of-domain inputs are handled, but hopefully no major breakage, and I'm optimistic that existing styles can be mechanically migrated.\r\n\r\n## Renderer changes\r\n\r\nThe major rendering change will be adjusting to the fact that rendering properties will become per-feature (and therefore per-vertex) attributes when a data-valued scale is used. For layout properties, this doesn't pose much of a challenge -- because some layout properties support token interpolation they are already assumed to be per-feature, the layout code already has direct access to feature data, and the output of the layout phase is GL buffers which are effectively static once calculated.\r\n\r\nFor paint properties, it's a different story:\r\n\r\n* The rendering thread does not currently have access to feature data. We'll need to start transferring it from the workers.\r\n* The values of paint properties are passed to the shader via uniform variables. We'll need to convert them to vertex attributes and provide corresponding vertex attribute buffers for data-valued scales. For efficiency we'll want to use `gl.vertexAttrib[1234][fv]` for `$zoom`-valued scales.\r\n* The values of paint properties are currently evaluated when the zoom-level or map classes change, and the results are then stored on a per-property basis. When data-valued scales are used, we'll need to evaluate them on a per-feature basis and store them on a per-vertex basis instead. However, we can reuse the results across zoom level changes that don't cross an integer boundary. And again, for efficiency we'll want to continue to use the current per-property evaluation and storage for `$zoom`-valued scales.\r\n\r\nThe performance impact of data-valued paint properties will need to be carefully monitored, but in @lucaswoj's prototype performance is encouragingly good.\r\n\r\n## Sequencing work\r\n\r\nHere's what I'm seeing:\r\n\r\n1. Kick off a v8 style spec. Make spec changes to convert existing `interpolated` and `piecewise-constant` function types to `linear`, `pow`, and `quantize` scales, along with corresponding migrations and style updates. But support only `$zoom`-valued scales.\r\n2. Rewrite mapbox-gl-function as mapbox-gl-scale, with support for those types, and integrate into mapbox-gl-js.\r\n3. Convert all necessary shader uniforms to attributes set via `gl.vertexAttrib[1234][fv]`.\r\n\r\nThat gets us no changes in effective functionality, but the right base to begin adding it.\r\n\r\n4. Implement `ordinal` scale type and data-valued scales in mapbox-gl-scale.\r\n4. Implement support for them in layout properties.\r\n4. Change worker to pass feature data (indexed by vertex) back to main thread.\r\n5. Implement per-feature evaluation and per-vertex storage for paint properties. (Biggest unknown.)\r\n6. Port it all to native.\r\n\r\n## Out of scope\r\n\r\nHere's what we _aren't_ going to do, at least right now:\r\n\r\n* Support arbitrary arithmetic expressions (#47). While it's possible to imagine an `\"expression\"` scale type defined by a grammar similar to [filters](https://www.mapbox.com/mapbox-gl-style-spec/#filter), I want to see if it's really necessary.\r\n* Scales based on _multiple_ input values. This would probably require arbitrary arithmetic expressions.\r\n* \"tokens in number properties\" (#105). I think embedding tokens in strings would be problematic for numeric values, and scales should give us all the needed functionality anyway.\r\n"},"comment":{"url":"https://api.github.com/repos/mapbox/mapbox-gl-style-spec/issues/comments/145177246","html_url":"https://github.com/mapbox/mapbox-gl-style-spec/issues/249#issuecomment-145177246","issue_url":"https://api.github.com/repos/mapbox/mapbox-gl-style-spec/issues/249","id":145177246,"user":{"login":"tmcw","id":32314,"avatar_url":"https://avatars.githubusercontent.com/u/32314?v=3","gravatar_id":"","url":"https://api.github.com/users/tmcw","html_url":"https://github.com/tmcw","followers_url":"https://api.github.com/users/tmcw/followers","following_url":"https://api.github.com/users/tmcw/following{/other_user}","gists_url":"https://api.github.com/users/tmcw/gists{/gist_id}","starred_url":"https://api.github.com/users/tmcw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tmcw/subscriptions","organizations_url":"https://api.github.com/users/tmcw/orgs","repos_url":"https://api.github.com/users/tmcw/repos","events_url":"https://api.github.com/users/tmcw/events{/privacy}","received_events_url":"https://api.github.com/users/tmcw/received_events","type":"User","site_admin":false},"created_at":"2015-10-02T23:00:55Z","updated_at":"2015-10-02T23:00:55Z","body":"@stvno not entirely: unlike untiled date, we can't rely on just feature indexes to join data. But we're tinkering a lot right now with joining data to existing vector tiles based on stable IDs, like OSM IDs or something similar. Initially we might do so at the server level, but I imagine it should be implemented on both levels so you can also dynamically join tabular data to features in the client."}}