{"action":"created","issue":{"url":"https://api.github.com/repos/borgbackup/borg/issues/452","labels_url":"https://api.github.com/repos/borgbackup/borg/issues/452/labels{/name}","comments_url":"https://api.github.com/repos/borgbackup/borg/issues/452/comments","events_url":"https://api.github.com/repos/borgbackup/borg/issues/452/events","html_url":"https://github.com/borgbackup/borg/pull/452","id":119799268,"number":452,"title":"misc. hash table tuning","user":{"login":"ThomasWaldmann","id":356103,"avatar_url":"https://avatars.githubusercontent.com/u/356103?v=3","gravatar_id":"","url":"https://api.github.com/users/ThomasWaldmann","html_url":"https://github.com/ThomasWaldmann","followers_url":"https://api.github.com/users/ThomasWaldmann/followers","following_url":"https://api.github.com/users/ThomasWaldmann/following{/other_user}","gists_url":"https://api.github.com/users/ThomasWaldmann/gists{/gist_id}","starred_url":"https://api.github.com/users/ThomasWaldmann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ThomasWaldmann/subscriptions","organizations_url":"https://api.github.com/users/ThomasWaldmann/orgs","repos_url":"https://api.github.com/users/ThomasWaldmann/repos","events_url":"https://api.github.com/users/ThomasWaldmann/events{/privacy}","received_events_url":"https://api.github.com/users/ThomasWaldmann/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-12-01T20:19:24Z","updated_at":"2015-12-01T20:36:36Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/borgbackup/borg/pulls/452","html_url":"https://github.com/borgbackup/borg/pull/452","diff_url":"https://github.com/borgbackup/borg/pull/452.diff","patch_url":"https://github.com/borgbackup/borg/pull/452.patch"},"body":"BUCKET_UPPER_LIMIT: 90% load degrades hash table performance severely,\r\nso I lowered that to 75% (which is a usual value - java uses 75%, python uses 66%).\r\nI chose the higher value of both because we also should not consume too much\r\nmemory, considering the RAM usage already is rather high.\r\n\r\nMIN_BUCKETS: I can't explain why, but benchmarks showed that choosing 2^N as\r\ntable size severely degrades performance (by 3 orders of magnitude!). So a prime\r\nstart value improves this a lot, even if we later still use the grow-by-2x algorithm.\r\n\r\nhashindex_resize: removed the hashindex_get() call as we already know that the values\r\ncome at key + key_size address.\r\n\r\nhashindex_init: do not calloc X*Y elements of size 1, but rather X elements of size Y.\r\nMakes the code simpler, not sure if it affects performance.\r\n\r\nThe tests needed fixing as the resulting hashtable blob is now of course different due\r\nto the above changes, so its sha hash changed."},"comment":{"url":"https://api.github.com/repos/borgbackup/borg/issues/comments/161088125","html_url":"https://github.com/borgbackup/borg/pull/452#issuecomment-161088125","issue_url":"https://api.github.com/repos/borgbackup/borg/issues/452","id":161088125,"user":{"login":"codecov-io","id":8655789,"avatar_url":"https://avatars.githubusercontent.com/u/8655789?v=3","gravatar_id":"","url":"https://api.github.com/users/codecov-io","html_url":"https://github.com/codecov-io","followers_url":"https://api.github.com/users/codecov-io/followers","following_url":"https://api.github.com/users/codecov-io/following{/other_user}","gists_url":"https://api.github.com/users/codecov-io/gists{/gist_id}","starred_url":"https://api.github.com/users/codecov-io/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/codecov-io/subscriptions","organizations_url":"https://api.github.com/users/codecov-io/orgs","repos_url":"https://api.github.com/users/codecov-io/repos","events_url":"https://api.github.com/users/codecov-io/events{/privacy}","received_events_url":"https://api.github.com/users/codecov-io/received_events","type":"User","site_admin":false},"created_at":"2015-12-01T20:36:36Z","updated_at":"2015-12-01T20:36:36Z","body":"## [Current coverage][1] is `83.36%`\n> Merging **#452** into **master** will increase coverage by **+0.72%** as of [`e983384`][3]\n\n\n```diff\n@@            master    #452   diff @@\n======================================\n  Files           13      13       \n  Stmts         3751    3751       \n  Branches       646     646       \n  Methods          0       0       \n======================================\n+ Hit           3100    3127    +27\n+ Partial        196     193     -3\n+ Missed         455     431    -24\n```\n\n> Review entire [Coverage Diff][4] as of [`e983384`][3]\n\n\n-----\n### [Uncovered Suggestions][2]\n\n1. `+0.40%` via [borg/helpers.py#738...752](https://codecov.io/github/borgbackup/borg/borg/helpers.py?ref=e983384b16fba9072dc0a64502fecfd36ec29567#738) \n1. `+0.32%` via [borg/archiver.py#366...377](https://codecov.io/github/borgbackup/borg/borg/archiver.py?ref=e983384b16fba9072dc0a64502fecfd36ec29567#366) \n1. `+0.29%` via [borg/key.py#407...417](https://codecov.io/github/borgbackup/borg/borg/key.py?ref=e983384b16fba9072dc0a64502fecfd36ec29567#407) \n1. *[See 7 more...][2]*\n\n[1]: https://codecov.io/github/borgbackup/borg?ref=e983384b16fba9072dc0a64502fecfd36ec29567\n[2]: https://codecov.io/github/borgbackup/borg/features/suggestions?ref=e983384b16fba9072dc0a64502fecfd36ec29567\n[3]: https://codecov.io/github/borgbackup/borg/commit/e983384b16fba9072dc0a64502fecfd36ec29567\n[4]: https://codecov.io/github/borgbackup/borg/compare/d1377e5e151316b539afe2d9220a8686e6618046...e983384b16fba9072dc0a64502fecfd36ec29567\n\n> Powered by [Codecov](https://codecov.io). Updated on successful CI builds."}}