{"action":"created","issue":{"url":"https://api.github.com/repos/tsenart/vegeta/issues/129","labels_url":"https://api.github.com/repos/tsenart/vegeta/issues/129/labels{/name}","comments_url":"https://api.github.com/repos/tsenart/vegeta/issues/129/comments","events_url":"https://api.github.com/repos/tsenart/vegeta/issues/129/events","html_url":"https://github.com/tsenart/vegeta/issues/129","id":98213950,"number":129,"title":"Quantiles are incorrectly calculated","user":{"login":"alexeyrodriguez","id":3804074,"avatar_url":"https://avatars.githubusercontent.com/u/3804074?v=3","gravatar_id":"","url":"https://api.github.com/users/alexeyrodriguez","html_url":"https://github.com/alexeyrodriguez","followers_url":"https://api.github.com/users/alexeyrodriguez/followers","following_url":"https://api.github.com/users/alexeyrodriguez/following{/other_user}","gists_url":"https://api.github.com/users/alexeyrodriguez/gists{/gist_id}","starred_url":"https://api.github.com/users/alexeyrodriguez/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexeyrodriguez/subscriptions","organizations_url":"https://api.github.com/users/alexeyrodriguez/orgs","repos_url":"https://api.github.com/users/alexeyrodriguez/repos","events_url":"https://api.github.com/users/alexeyrodriguez/events{/privacy}","received_events_url":"https://api.github.com/users/alexeyrodriguez/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-07-30T16:45:09Z","updated_at":"2015-07-31T22:19:32Z","closed_at":null,"body":"In some executions of vegeta the metrics command generates incorrect quantiles, namely the value of the 99th quantile is the same as max, which is wrong. I double checked by dumping the results file into a CSV and calculating the quantiles using R. I'll be happy to provide the offending binary file (I see no way to attach it here)."},"comment":{"url":"https://api.github.com/repos/tsenart/vegeta/issues/comments/126826036","html_url":"https://github.com/tsenart/vegeta/issues/129#issuecomment-126826036","issue_url":"https://api.github.com/repos/tsenart/vegeta/issues/129","id":126826036,"user":{"login":"tsenart","id":67471,"avatar_url":"https://avatars.githubusercontent.com/u/67471?v=3","gravatar_id":"","url":"https://api.github.com/users/tsenart","html_url":"https://github.com/tsenart","followers_url":"https://api.github.com/users/tsenart/followers","following_url":"https://api.github.com/users/tsenart/following{/other_user}","gists_url":"https://api.github.com/users/tsenart/gists{/gist_id}","starred_url":"https://api.github.com/users/tsenart/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tsenart/subscriptions","organizations_url":"https://api.github.com/users/tsenart/orgs","repos_url":"https://api.github.com/users/tsenart/repos","events_url":"https://api.github.com/users/tsenart/events{/privacy}","received_events_url":"https://api.github.com/users/tsenart/received_events","type":"User","site_admin":false},"created_at":"2015-07-31T22:19:32Z","updated_at":"2015-07-31T22:19:32Z","body":"Vegeta is using github.com/bmizerany/perks/quantile for quantile calculation. As paraphrased below, this library trades accuracy for guaranteed low memory and CPU bounds over unbounded data streams.\r\n\r\nNow, it might be worth to re-evaluate if we need those properties since the percentiles are computed in batch, not over a stream as they once were supposed to be.\r\n\r\nWhat would your advice be on the matter? Thanks for reporting this. :-)\r\n\r\n```go\r\n// Package quantile computes approximate quantiles over an unbounded data\r\n// stream within low memory and CPU bounds.\r\n//\r\n// A small amount of accuracy is traded to achieve the above properties.\r\n//\r\n// Multiple streams can be merged before calling Query to generate a single set\r\n// of results. This is meaningful when the streams represent the same type of\r\n// data. See Merge and Samples.\r\n//\r\n// For more detailed information about the algorithm used, see:\r\n//\r\n// Effective Computation of Biased Quantiles over Data Streams\r\n//\r\n// http://www.cs.rutgers.edu/~muthu/bquant.pdf\r\n```"}}