{"action":"created","issue":{"url":"https://api.github.com/repos/inaturalist/inaturalist/issues/601","labels_url":"https://api.github.com/repos/inaturalist/inaturalist/issues/601/labels{/name}","comments_url":"https://api.github.com/repos/inaturalist/inaturalist/issues/601/comments","events_url":"https://api.github.com/repos/inaturalist/inaturalist/issues/601/events","html_url":"https://github.com/inaturalist/inaturalist/issues/601","id":88769449,"number":601,"title":"Replace Windshaft tiles with Ellasticsearch-based maps","user":{"login":"pleary","id":48566,"avatar_url":"https://avatars.githubusercontent.com/u/48566?v=3","gravatar_id":"","url":"https://api.github.com/users/pleary","html_url":"https://github.com/pleary","followers_url":"https://api.github.com/users/pleary/followers","following_url":"https://api.github.com/users/pleary/following{/other_user}","gists_url":"https://api.github.com/users/pleary/gists{/gist_id}","starred_url":"https://api.github.com/users/pleary/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pleary/subscriptions","organizations_url":"https://api.github.com/users/pleary/orgs","repos_url":"https://api.github.com/users/pleary/repos","events_url":"https://api.github.com/users/pleary/events{/privacy}","received_events_url":"https://api.github.com/users/pleary/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/inaturalist/inaturalist/labels/features","name":"features","color":"fbca04"},{"url":"https://api.github.com/repos/inaturalist/inaturalist/labels/in%20progress","name":"in progress","color":"ededed"}],"state":"open","locked":false,"assignee":{"login":"pleary","id":48566,"avatar_url":"https://avatars.githubusercontent.com/u/48566?v=3","gravatar_id":"","url":"https://api.github.com/users/pleary","html_url":"https://github.com/pleary","followers_url":"https://api.github.com/users/pleary/followers","following_url":"https://api.github.com/users/pleary/following{/other_user}","gists_url":"https://api.github.com/users/pleary/gists{/gist_id}","starred_url":"https://api.github.com/users/pleary/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pleary/subscriptions","organizations_url":"https://api.github.com/users/pleary/orgs","repos_url":"https://api.github.com/users/pleary/repos","events_url":"https://api.github.com/users/pleary/events{/privacy}","received_events_url":"https://api.github.com/users/pleary/received_events","type":"User","site_admin":false},"milestone":null,"comments":17,"created_at":"2015-06-16T16:06:21Z","updated_at":"2015-06-17T06:03:14Z","closed_at":null,"body":"Windshaft tiles use the DB for querying, and this can cause slow performance when we need to generate tiles for anything that has not been denormalized. Here are some pros t switching to ES-based tiles:\r\n\r\n - We will no longer need to generate cache tables for grid map zoom level (and as a result the low-zoom tiles will no longer be limited to data generated at denormalization time - they will be based on real-time data)\r\n - Elasticsearch uses a geohash indexing system internally, which essentially gives use the ability to efficiently generate low-zoom tiles representing 10s of thousands of points\r\n - We will be able render tiles on demand for queries based on any field in Elasticsearch (as long as the code has been updated to support it). That means we can generate map tiles for just about any query generated from the form at http://www.inaturalist.org/observations/\r\n - Elasticsearch can generate real-time replicas in different data center, so we can host a copy at GoGrid and run our entire map tiling code there. That will allow us to get rid of two small Rackspace servers currently hosting Windshaft\r\n - The same code we use to generate queries for map tiles can be used in a data API that can one day replace a lot of observations API logic in the Rails codebase. This will also make new development easier, like getting into websocket-based features\r\n\r\nI have my latest proposal for the first round design of the maps running on http://gonatus.inaturalist.org (same authentication as Staging). See https://github.com/inaturalist/design/issues/14 for more design discussion. Here is a summary of all changes compared to our Windshaft implementation:\r\n\r\n - All grids are replaced by dots with opacity 0.4, but using the same color schemes as grids use. So dots on http://gonatus.inaturalist.org/observations are all grey, and dots on http://gonatus.inaturalist.org/taxa/3/map are all blue. The transparency allows at least a minor level of representing point density\r\n - The high-zoom tiles (formerly grids) are also generating interactivity data so all map dots are clickable and open the corresponding observation. If there are many in a single interactivity grid cell, then the most recent will be shown\r\n - Not all filters have been implemented yet, but we can currently do place filtering with map tiles which we had to disable in Windshaft http://gonatus.inaturalist.org/observations?place_id=6793\r\n - I added a heatmap view of observations http://gonatus.inaturalist.org/observations/map (clickable like everything else)\r\n - It's worth noting that the Windshaft grids were more visible on Google's satellite maps since they had a white border, and the new dots have border because I thought it made high-zoom maps too noisy\r\n\r\nI have gonatus accepting some extra parameters for testing:\r\n\r\nTo change the zoom where we cross over from summary dots to real points (it is currently set at zoom 9, the same as Windshaft, but we can set it to whatever we want):\r\n  http://gonatus.inaturalist.org/observations/map?gridmaxzoom=3\r\n\r\nTo try different color of the dots (%23000000 is #000000 escaped):\r\n  http://gonatus.inaturalist.org/observations/map?color=%23000000\r\n\r\nMy proposal is to release maps as seen on gonatus, with clickable grey or colored dots instead of grids. We can always change the styles later, but I am basically looking for confirmation we're happy with this as a first pass so that we can move off of Windshaft. It would be great of we could add any comments about changes wanted/needed before release of these maps to this ticket. Then any comments about future maps design directions can go to https://github.com/inaturalist/design/issues/14 , and any comments about future functionality can be added to the repo for the new maps https://github.com/inaturalist/iNaturalistAPI "},"comment":{"url":"https://api.github.com/repos/inaturalist/inaturalist/issues/comments/112664586","html_url":"https://github.com/inaturalist/inaturalist/issues/601#issuecomment-112664586","issue_url":"https://api.github.com/repos/inaturalist/inaturalist/issues/601","id":112664586,"user":{"login":"kueda","id":22691,"avatar_url":"https://avatars.githubusercontent.com/u/22691?v=3","gravatar_id":"","url":"https://api.github.com/users/kueda","html_url":"https://github.com/kueda","followers_url":"https://api.github.com/users/kueda/followers","following_url":"https://api.github.com/users/kueda/following{/other_user}","gists_url":"https://api.github.com/users/kueda/gists{/gist_id}","starred_url":"https://api.github.com/users/kueda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kueda/subscriptions","organizations_url":"https://api.github.com/users/kueda/orgs","repos_url":"https://api.github.com/users/kueda/repos","events_url":"https://api.github.com/users/kueda/events{/privacy}","received_events_url":"https://api.github.com/users/kueda/received_events","type":"User","site_admin":false},"created_at":"2015-06-17T06:03:14Z","updated_at":"2015-06-17T06:03:14Z","body":"I think the pseudo heatmap with white border does a decent job with visibility for maps with a lot of points, but not so much when viewing small amounts of data with little spatial spread from very high zoom levels, e.g.\r\n\r\nhttp://gonatus.inaturalist.org/observations/map?place_id=5586&heatmap_colors=green,white0.5#3/32.314/-75.101\r\n\r\nIf you turn off the place boundary, you get a white smudge which is hard to see with the satellite tiles and almost impossible to see over the terrain and road tiles.\r\n\r\nHere's how the Cascades Panthea Moth looks [with the rainbow heatmap](http://www.inaturalist.org/observations/map?taxon_id=226911&heatmap=true&elastic=true#5/41.968/-118.828):\r\n\r\n![screen shot 2015-06-16 at 10 55 23 pm](https://cloud.githubusercontent.com/assets/22691/8200512/e03cc1ce-147a-11e5-8b3d-37a727788ec3.png)\r\n\r\nAnd here's [the same map with points](http://gonatus.inaturalist.org/observations/map?taxon_id=226911#5/41.03/-120.388):\r\n\r\n![screen shot 2015-06-16 at 11 02 11 pm](https://cloud.githubusercontent.com/assets/22691/8200578/c20752ae-147b-11e5-8b22-82aa2d50c324.png)\r\n\r\nThere are aesthetics at play here, of course, but I think the hard-to-see problem is still a legit issue with the heatmaps."}}