{"action":"created","issue":{"url":"https://api.github.com/repos/graphite-project/ceres/issues/32","labels_url":"https://api.github.com/repos/graphite-project/ceres/issues/32/labels{/name}","comments_url":"https://api.github.com/repos/graphite-project/ceres/issues/32/comments","events_url":"https://api.github.com/repos/graphite-project/ceres/issues/32/events","html_url":"https://github.com/graphite-project/ceres/pull/32","id":82481878,"number":32,"title":"(Another) fixed reading archived data","user":{"login":"ibuclaw","id":397929,"avatar_url":"https://avatars.githubusercontent.com/u/397929?v=3","gravatar_id":"","url":"https://api.github.com/users/ibuclaw","html_url":"https://github.com/ibuclaw","followers_url":"https://api.github.com/users/ibuclaw/followers","following_url":"https://api.github.com/users/ibuclaw/following{/other_user}","gists_url":"https://api.github.com/users/ibuclaw/gists{/gist_id}","starred_url":"https://api.github.com/users/ibuclaw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ibuclaw/subscriptions","organizations_url":"https://api.github.com/users/ibuclaw/orgs","repos_url":"https://api.github.com/users/ibuclaw/repos","events_url":"https://api.github.com/users/ibuclaw/events{/privacy}","received_events_url":"https://api.github.com/users/ibuclaw/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-05-29T15:52:23Z","updated_at":"2015-06-03T16:36:59Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/graphite-project/ceres/pulls/32","html_url":"https://github.com/graphite-project/ceres/pull/32","diff_url":"https://github.com/graphite-project/ceres/pull/32.diff","patch_url":"https://github.com/graphite-project/ceres/pull/32.patch"},"body":"Similar to #10 but differs in the following ways:\r\n\r\n- Don't blindly aggregate to the largest timestep in the range read.  I found this to be a massive problem when you are pulling metrics from nodes that have been recording for different amounts of time  (ie: Request 3 months, but one node only has data for 3 days)\r\n\r\nWhat I found better was to start at the default timeStep, then aggregate as needed when you reach some magic high number.  This ended up being the number of retentions held for the timeStep you are normalizing to.  One litmus test I used was: how fast is it to retrieve 3000 days of data? Is that faster or slow than 300 days? 30 days? 3 days? 3 hours?  And had comparable results at either end of the spectrum based on my retention configuration.\r\n\r\n- Supporting other kinds of aggregation that carbon may set, defaulting to 'average'.  This matches in functionality something similar to that of Whisper.\r\n\r\nI'd also like to find a way to re-factor the repeated parts into some re-useable function, but stopped when the number of passed parameters exceeded five."},"comment":{"url":"https://api.github.com/repos/graphite-project/ceres/issues/comments/108515852","html_url":"https://github.com/graphite-project/ceres/pull/32#issuecomment-108515852","issue_url":"https://api.github.com/repos/graphite-project/ceres/issues/32","id":108515852,"user":{"login":"ibuclaw","id":397929,"avatar_url":"https://avatars.githubusercontent.com/u/397929?v=3","gravatar_id":"","url":"https://api.github.com/users/ibuclaw","html_url":"https://github.com/ibuclaw","followers_url":"https://api.github.com/users/ibuclaw/followers","following_url":"https://api.github.com/users/ibuclaw/following{/other_user}","gists_url":"https://api.github.com/users/ibuclaw/gists{/gist_id}","starred_url":"https://api.github.com/users/ibuclaw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ibuclaw/subscriptions","organizations_url":"https://api.github.com/users/ibuclaw/orgs","repos_url":"https://api.github.com/users/ibuclaw/repos","events_url":"https://api.github.com/users/ibuclaw/events{/privacy}","received_events_url":"https://api.github.com/users/ibuclaw/received_events","type":"User","site_admin":false},"created_at":"2015-06-03T16:36:59Z","updated_at":"2015-06-03T16:36:59Z","body":"> Thinking aloud: are we relying on carbon's storage-schema validation to enforce retentions being evenly divisible?\r\nTBC, I dont think we should do any validation here but if it's not done somewhere I expect we'll eventually see some weird bug reports when people configure retentions unusually\r\n\r\nWhisper documents this: http://graphite.readthedocs.org/en/latest/whisper.html#archives-retention-and-precision\r\n\r\nSo I assume that they use similar assumptions too.  If there is a better way, I'd not object to it though."}}