{"action":"created","issue":{"url":"https://api.github.com/repos/graphite-project/whisper/issues/109","labels_url":"https://api.github.com/repos/graphite-project/whisper/issues/109/labels{/name}","comments_url":"https://api.github.com/repos/graphite-project/whisper/issues/109/comments","events_url":"https://api.github.com/repos/graphite-project/whisper/issues/109/events","html_url":"https://github.com/graphite-project/whisper/pull/109","id":54066009,"number":109,"title":"Implement configurable fetch strategy: coarse","user":{"login":"jraby","id":2365261,"avatar_url":"https://avatars.githubusercontent.com/u/2365261?v=3","gravatar_id":"","url":"https://api.github.com/users/jraby","html_url":"https://github.com/jraby","followers_url":"https://api.github.com/users/jraby/followers","following_url":"https://api.github.com/users/jraby/following{/other_user}","gists_url":"https://api.github.com/users/jraby/gists{/gist_id}","starred_url":"https://api.github.com/users/jraby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jraby/subscriptions","organizations_url":"https://api.github.com/users/jraby/orgs","repos_url":"https://api.github.com/users/jraby/repos","events_url":"https://api.github.com/users/jraby/events{/privacy}","received_events_url":"https://api.github.com/users/jraby/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/graphite-project/whisper/labels/question","name":"question","color":"cc317c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-01-12T15:27:28Z","updated_at":"2015-06-17T22:37:45Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/graphite-project/whisper/pulls/109","html_url":"https://github.com/graphite-project/whisper/pull/109","diff_url":"https://github.com/graphite-project/whisper/pull/109.diff","patch_url":"https://github.com/graphite-project/whisper/pull/109.patch"},"body":"The idea is to return datapoints from the coarsest archive that has above _threshold_ datapoints in the requested timespan.\r\nIf such an archive is not available, it returns the first archive matching the timespan. (as without this patch)\r\n\r\nWith appropriate retention periods and thresholds configuration, using this fetch strategy can yield a major speed increase, since much less datapoints need to be processed.\r\n(sent to a frontend and/or rendered into a graph)\r\n\r\nThe caller of fetch() will need to set FETCH_STRATEGY to 'coarse' and FETCH_STRATEGY_COARSE_THRESHOLD to the desired minimum datapoints.\r\nThe default strategy is unchanged and the default threshold is equivalent to the default fetch strategy.\r\n\r\nWe've been running this on `0.9.x` for more than 6 months now with great success.\r\nIt allows us to keep the most precise data for a long time (accessible by zooming in the graphs), and be able to generate weekly/monthly graphs quickly.\r\nFor example, when running this with a 330 threshold, and `10s:7d,60s:30d` retentions, whisper will use the 60s precision archive if the requests spans more than 5.5h.\r\n \r\nKeep in mind that the behaviour of the graph will change in a meaningful way:  for large time ranges, whisper will return aggregated data and thus 'cut' spikes.\r\n\r\nWhat do you guys think of that concept?"},"comment":{"url":"https://api.github.com/repos/graphite-project/whisper/issues/comments/112971557","html_url":"https://github.com/graphite-project/whisper/pull/109#issuecomment-112971557","issue_url":"https://api.github.com/repos/graphite-project/whisper/issues/109","id":112971557,"user":{"login":"jraby","id":2365261,"avatar_url":"https://avatars.githubusercontent.com/u/2365261?v=3","gravatar_id":"","url":"https://api.github.com/users/jraby","html_url":"https://github.com/jraby","followers_url":"https://api.github.com/users/jraby/followers","following_url":"https://api.github.com/users/jraby/following{/other_user}","gists_url":"https://api.github.com/users/jraby/gists{/gist_id}","starred_url":"https://api.github.com/users/jraby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jraby/subscriptions","organizations_url":"https://api.github.com/users/jraby/orgs","repos_url":"https://api.github.com/users/jraby/repos","events_url":"https://api.github.com/users/jraby/events{/privacy}","received_events_url":"https://api.github.com/users/jraby/received_events","type":"User","site_admin":false},"created_at":"2015-06-17T22:37:45Z","updated_at":"2015-06-17T22:37:45Z","body":"@SEJeff I'm all for it, I actually have a few use cases for that here.\r\nThe implementation in this PR is a bit of a big (huge) hammer, it would be nice to be able to select the fetch strategy from the render requests.\r\n\r\nHowever, I don't have time to implement this at the moment.\r\nWhen I originally implemented this, I didn't find an easy way to pass arguments from the render request down to the whisper module, maybe I'll see something when I look at this again."}}