{"action":"created","issue":{"url":"https://api.github.com/repos/mozilla/ichnaea/issues/421","labels_url":"https://api.github.com/repos/mozilla/ichnaea/issues/421/labels{/name}","comments_url":"https://api.github.com/repos/mozilla/ichnaea/issues/421/comments","events_url":"https://api.github.com/repos/mozilla/ichnaea/issues/421/events","html_url":"https://github.com/mozilla/ichnaea/issues/421","id":64544235,"number":421,"title":"Implement fallback location provider","user":{"login":"hannosch","id":483109,"avatar_url":"https://avatars.githubusercontent.com/u/483109?v=3","gravatar_id":"","url":"https://api.github.com/users/hannosch","html_url":"https://github.com/hannosch","followers_url":"https://api.github.com/users/hannosch/followers","following_url":"https://api.github.com/users/hannosch/following{/other_user}","gists_url":"https://api.github.com/users/hannosch/gists{/gist_id}","starred_url":"https://api.github.com/users/hannosch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hannosch/subscriptions","organizations_url":"https://api.github.com/users/hannosch/orgs","repos_url":"https://api.github.com/users/hannosch/repos","events_url":"https://api.github.com/users/hannosch/events{/privacy}","received_events_url":"https://api.github.com/users/hannosch/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/mozilla/ichnaea/labels/idea","name":"idea","color":"0052cc"}],"state":"open","locked":false,"assignee":{"login":"jaredkerim","id":119884,"avatar_url":"https://avatars.githubusercontent.com/u/119884?v=3","gravatar_id":"","url":"https://api.github.com/users/jaredkerim","html_url":"https://github.com/jaredkerim","followers_url":"https://api.github.com/users/jaredkerim/followers","following_url":"https://api.github.com/users/jaredkerim/following{/other_user}","gists_url":"https://api.github.com/users/jaredkerim/gists{/gist_id}","starred_url":"https://api.github.com/users/jaredkerim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaredkerim/subscriptions","organizations_url":"https://api.github.com/users/jaredkerim/orgs","repos_url":"https://api.github.com/users/jaredkerim/repos","events_url":"https://api.github.com/users/jaredkerim/events{/privacy}","received_events_url":"https://api.github.com/users/jaredkerim/received_events","type":"User","site_admin":false},"milestone":null,"comments":1,"created_at":"2015-03-26T14:46:28Z","updated_at":"2015-05-07T12:21:18Z","closed_at":null,"body":"We should add support for querying a secondary fallback location provider, if we aren't satisfied with our own location result.\r\n\r\nThis includes:\r\n- Add new settings to the config file, maybe fallback_url, fallback_apikey, fallback_ratelimit, fallback_cache_timeout (could make this into a separate section in the ini file, but the code reading the file makes that a bit harder)\r\n- Add new column to the api_key table, a boolean fallback_allowed\r\n- Add a new location provider doing a \"geolocate\" type query to the fallback provider if the current api key is allowed to do so\r\n- Add rate limitation logic, only allow <fallback_ratelimit config option> number of requests, potentially reusing / extending the code in ichnaea.service.base.rate_limit. The config option might have to be a simple expression like \"5/5s\" stating \"allow 5 requests per 5 second interval\", only supporting seconds as the time interval. The existing rate_limit function disables rate limitation if redis is not available. For outbound requests we want to do the opposite and not do a request if redis is unavailable.\r\n- In the first round, only do a fallback request if we couldn't get any non-geoip data from our own database. Also never do a fallback request if the request contained no valid cell/wifi identifiers (this includes only getting a single wifi data point). If all we could get is a geoip result, there's no value in doing a fallback.\r\n- In the fallback query, use the custom syntax and add`\"fallbacks\": {\"lacf\": 1}` to the geolocate query (see http://combain.com/api/ fallbacks or the WCDMA lacf example. This should match what we are doing.\r\n- Add caching of the responses we got from the fallback (in Redis). Use the fallback_cache_timeout config setting to determine the cache timeout value. The config option can be specified as an integer in seconds. At first only do response caching if we get a query containing a single cell network. The cache key can then consist of some cache prefix and the cell hash key. As a value we can store the actual JSON response (lat/lon/accuracy) or some token to cache not found responses.\r\n\r\nStill undefined:\r\n- What exact metrics should we gather? Something about how many fallback requests we make, how many times do we reach the rate limit. How does the fallback tie into our KPI metrics (the provider group stuff), as we cannot reliably distinguish between getting a cell or wifi based response from the fallback. We also want some metrics on cache effectiveness / hit rates.\r\n- Do we want to do caching for Wifi based queries? A cache key consisting of all bssid's could get really large. For queries based on multiple data points the signal strength data also contributes significantly to a better position estimate, so we might need to make that part of the cache key as well. Might lead to cache key explosion without any cache hits. Probably defer to later."},"comment":{"url":"https://api.github.com/repos/mozilla/ichnaea/issues/comments/99842897","html_url":"https://github.com/mozilla/ichnaea/issues/421#issuecomment-99842897","issue_url":"https://api.github.com/repos/mozilla/ichnaea/issues/421","id":99842897,"user":{"login":"hannosch","id":483109,"avatar_url":"https://avatars.githubusercontent.com/u/483109?v=3","gravatar_id":"","url":"https://api.github.com/users/hannosch","html_url":"https://github.com/hannosch","followers_url":"https://api.github.com/users/hannosch/followers","following_url":"https://api.github.com/users/hannosch/following{/other_user}","gists_url":"https://api.github.com/users/hannosch/gists{/gist_id}","starred_url":"https://api.github.com/users/hannosch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hannosch/subscriptions","organizations_url":"https://api.github.com/users/hannosch/orgs","repos_url":"https://api.github.com/users/hannosch/repos","events_url":"https://api.github.com/users/hannosch/events{/privacy}","received_events_url":"https://api.github.com/users/hannosch/received_events","type":"User","site_admin":false},"created_at":"2015-05-07T12:21:18Z","updated_at":"2015-05-07T12:21:18Z","body":"What remains of this are:\r\n* In the fallback query, use the custom syntax and add\"fallbacks\": {\"lacf\": 1} to the geolocate query (see http://combain.com/api/ fallbacks or the WCDMA lacf example. This should match what we are doing.\r\n* Finish the rate limit (#432)\r\n* Add caching of single cell queries\r\n* Simplify metrics names (#430)\r\n* Add new metrics (#433)"}}