{"action":"closed","issue":{"url":"https://api.github.com/repos/lemire/RoaringBitmap/issues/58","labels_url":"https://api.github.com/repos/lemire/RoaringBitmap/issues/58/labels{/name}","comments_url":"https://api.github.com/repos/lemire/RoaringBitmap/issues/58/comments","events_url":"https://api.github.com/repos/lemire/RoaringBitmap/issues/58/events","html_url":"https://github.com/lemire/RoaringBitmap/issues/58","id":121276859,"number":58,"title":"Question: Benchmark tests","user":{"login":"Tornhoof","id":7989549,"avatar_url":"https://avatars.githubusercontent.com/u/7989549?v=3","gravatar_id":"","url":"https://api.github.com/users/Tornhoof","html_url":"https://github.com/Tornhoof","followers_url":"https://api.github.com/users/Tornhoof/followers","following_url":"https://api.github.com/users/Tornhoof/following{/other_user}","gists_url":"https://api.github.com/users/Tornhoof/gists{/gist_id}","starred_url":"https://api.github.com/users/Tornhoof/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Tornhoof/subscriptions","organizations_url":"https://api.github.com/users/Tornhoof/orgs","repos_url":"https://api.github.com/users/Tornhoof/repos","events_url":"https://api.github.com/users/Tornhoof/events{/privacy}","received_events_url":"https://api.github.com/users/Tornhoof/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-12-09T15:55:07Z","updated_at":"2015-12-09T23:59:36Z","closed_at":"2015-12-09T23:59:36Z","body":"Hello,\r\nas I'm currently in the process of porting RoaringBitmaps to C#, I'm also porting those real data tests.\r\nWhile my implementation has the same output cardinality as the unit tests, I can't really follow the time results of those tests.\r\nFor example:\r\n````\r\n  <testcase name=\"test[dataset=census-income, type=roaring, immutable=false]\" classname=\"org.roaringbitmap.realdata.RealDataBenchmarkOrTest\" time=\"1.469\"/>\r\n  <testcase name=\"test[dataset=census-income, type=roaring, immutable=true]\" classname=\"org.roaringbitmap.realdata.RealDataBenchmarkOrTest\" time=\"3.822\"/>\r\n````\r\nThe values appear to be too high, do they include the ZIP file part (loading and parsing the census data) or are there multiple iterations to the actual test. It uses JMH, which should handle all that warmup etc. stuff, but as I haven't done any Java work for over ten years I do not really know how e.g. the @Benchmark attribute works.\r\n\r\nAre there any real performance benchmarks against the real data, which only handle the actual bitwise operations?\r\n"}}