{"action":"created","issue":{"url":"https://api.github.com/repos/raintank/raintank-metric/issues/21","labels_url":"https://api.github.com/repos/raintank/raintank-metric/issues/21/labels{/name}","comments_url":"https://api.github.com/repos/raintank/raintank-metric/issues/21/comments","events_url":"https://api.github.com/repos/raintank/raintank-metric/issues/21/events","html_url":"https://github.com/raintank/raintank-metric/issues/21","id":104510112,"number":21,"title":"rollups.","user":{"login":"Dieterbe","id":20774,"avatar_url":"https://avatars.githubusercontent.com/u/20774?v=3","gravatar_id":"","url":"https://api.github.com/users/Dieterbe","html_url":"https://github.com/Dieterbe","followers_url":"https://api.github.com/users/Dieterbe/followers","following_url":"https://api.github.com/users/Dieterbe/following{/other_user}","gists_url":"https://api.github.com/users/Dieterbe/gists{/gist_id}","starred_url":"https://api.github.com/users/Dieterbe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Dieterbe/subscriptions","organizations_url":"https://api.github.com/users/Dieterbe/orgs","repos_url":"https://api.github.com/users/Dieterbe/repos","events_url":"https://api.github.com/users/Dieterbe/events{/privacy}","received_events_url":"https://api.github.com/users/Dieterbe/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-09-02T15:35:44Z","updated_at":"2015-09-03T09:28:50Z","closed_at":null,"body":"the time has come.\r\n\r\n* https://github.com/raintank/ops/issues/112 has some details\r\n* i know we wrote down some thoughts etc at the summit, do we still have those notes? or perhaps not that important\r\n* implementation will probably be an nsq consumer that generates all lower-res streams and stores them (including for current data), as opposed to a design where lower-res only starts where higher-res ends.\r\n* we can generate spread data like librato/omniti/hostedgraphite, or store individual min/max/avg/.. series, or use an algo like LTTB (see https://github.com/sveinn-steinarsson/flot-downsample/)"},"comment":{"url":"https://api.github.com/repos/raintank/raintank-metric/issues/comments/137390429","html_url":"https://github.com/raintank/raintank-metric/issues/21#issuecomment-137390429","issue_url":"https://api.github.com/repos/raintank/raintank-metric/issues/21","id":137390429,"user":{"login":"Dieterbe","id":20774,"avatar_url":"https://avatars.githubusercontent.com/u/20774?v=3","gravatar_id":"","url":"https://api.github.com/users/Dieterbe","html_url":"https://github.com/Dieterbe","followers_url":"https://api.github.com/users/Dieterbe/followers","following_url":"https://api.github.com/users/Dieterbe/following{/other_user}","gists_url":"https://api.github.com/users/Dieterbe/gists{/gist_id}","starred_url":"https://api.github.com/users/Dieterbe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Dieterbe/subscriptions","organizations_url":"https://api.github.com/users/Dieterbe/orgs","repos_url":"https://api.github.com/users/Dieterbe/repos","events_url":"https://api.github.com/users/Dieterbe/events{/privacy}","received_events_url":"https://api.github.com/users/Dieterbe/received_events","type":"User","site_admin":false},"created_at":"2015-09-03T09:28:50Z","updated_at":"2015-09-03T09:28:50Z","body":"> Querying this inMemory data is going to be faster then querying the TSDB, and so we should have graphite leverage it\r\n\r\nhow sure are we that there is a significant difference in response time for serving read requests for recent data? or maybe this question is not that relevant because if we're going to batch up writes (and it looks like we should) then we need this in-memory component to satisfy hot data anyway.\r\n\r\ni agree with your appoach, but the way i envision this, the nsq reading, inMemory store, and C* flushing can all happen in the same process, until we reach scalability limits."}}