{"action":"closed","number":146,"pull_request":{"url":"https://api.github.com/repos/citusdata/pg_shard/pulls/146","id":45371315,"html_url":"https://github.com/citusdata/pg_shard/pull/146","diff_url":"https://github.com/citusdata/pg_shard/pull/146.diff","patch_url":"https://github.com/citusdata/pg_shard/pull/146.patch","issue_url":"https://api.github.com/repos/citusdata/pg_shard/issues/146","number":146,"state":"closed","locked":false,"title":"Fix hash token distribution algorithm","user":{"login":"jasonmp85","id":50673,"avatar_url":"https://avatars.githubusercontent.com/u/50673?v=3","gravatar_id":"","url":"https://api.github.com/users/jasonmp85","html_url":"https://github.com/jasonmp85","followers_url":"https://api.github.com/users/jasonmp85/followers","following_url":"https://api.github.com/users/jasonmp85/following{/other_user}","gists_url":"https://api.github.com/users/jasonmp85/gists{/gist_id}","starred_url":"https://api.github.com/users/jasonmp85/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jasonmp85/subscriptions","organizations_url":"https://api.github.com/users/jasonmp85/orgs","repos_url":"https://api.github.com/users/jasonmp85/repos","events_url":"https://api.github.com/users/jasonmp85/events{/privacy}","received_events_url":"https://api.github.com/users/jasonmp85/received_events","type":"User","site_admin":false},"body":"There are `2^32` distinct \"hash tokens\" in our hash space, but we were using `INT32_MAX (2^32 - 1)` in the code instead. Because of this, shard counts which one might expect to divide evenly into the space (such as 16, 32, or 256) had fewer tokens than they should have. The remainder of the tokens were stuffed into the last shard, causing uneven load.\r\n\r\nThis fixes that by defining a 64-bit constant equal to `2^32` and then using that to calculate the distance between shards in the hash space.\r\n\r\nUnlike #145, in \"weird\" shard counts, the first `n-1` shards will have one size and the final shard will have _all_ leftover tokens. This means the final shard could contain up to `n-1` more hash tokens than the other shards, which might have load implications. In #145, all shards are within one token of each other.\r\n\r\nLook at the tests in each review to get an idea of the two approaches.","created_at":"2015-09-18T05:51:07Z","updated_at":"2015-10-05T06:51:19Z","closed_at":"2015-10-05T06:51:19Z","merged_at":"2015-10-05T06:51:19Z","merge_commit_sha":"33ba79f1f8195e389d598561bcb951588ec5a23d","assignee":{"login":"onderkalaci","id":7829470,"avatar_url":"https://avatars.githubusercontent.com/u/7829470?v=3","gravatar_id":"","url":"https://api.github.com/users/onderkalaci","html_url":"https://github.com/onderkalaci","followers_url":"https://api.github.com/users/onderkalaci/followers","following_url":"https://api.github.com/users/onderkalaci/following{/other_user}","gists_url":"https://api.github.com/users/onderkalaci/gists{/gist_id}","starred_url":"https://api.github.com/users/onderkalaci/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/onderkalaci/subscriptions","organizations_url":"https://api.github.com/users/onderkalaci/orgs","repos_url":"https://api.github.com/users/onderkalaci/repos","events_url":"https://api.github.com/users/onderkalaci/events{/privacy}","received_events_url":"https://api.github.com/users/onderkalaci/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/citusdata/pg_shard/pulls/146/commits","review_comments_url":"https://api.github.com/repos/citusdata/pg_shard/pulls/146/comments","review_comment_url":"https://api.github.com/repos/citusdata/pg_shard/pulls/comments{/number}","comments_url":"https://api.github.com/repos/citusdata/pg_shard/issues/146/comments","statuses_url":"https://api.github.com/repos/citusdata/pg_shard/statuses/50578da40b27de03b2d26aa9ed724e518a0dee4d","head":{"label":"citusdata:fix_hash_token_distribution","ref":"fix_hash_token_distribution","sha":"50578da40b27de03b2d26aa9ed724e518a0dee4d","user":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"repo":{"id":27522888,"name":"pg_shard","full_name":"citusdata/pg_shard","owner":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/citusdata/pg_shard","description":"PostgreSQL extension to scale out real-time reads and writes. Check out the mailing list at https://groups.google.com/group/pg_shard-users","fork":false,"url":"https://api.github.com/repos/citusdata/pg_shard","forks_url":"https://api.github.com/repos/citusdata/pg_shard/forks","keys_url":"https://api.github.com/repos/citusdata/pg_shard/keys{/key_id}","collaborators_url":"https://api.github.com/repos/citusdata/pg_shard/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/citusdata/pg_shard/teams","hooks_url":"https://api.github.com/repos/citusdata/pg_shard/hooks","issue_events_url":"https://api.github.com/repos/citusdata/pg_shard/issues/events{/number}","events_url":"https://api.github.com/repos/citusdata/pg_shard/events","assignees_url":"https://api.github.com/repos/citusdata/pg_shard/assignees{/user}","branches_url":"https://api.github.com/repos/citusdata/pg_shard/branches{/branch}","tags_url":"https://api.github.com/repos/citusdata/pg_shard/tags","blobs_url":"https://api.github.com/repos/citusdata/pg_shard/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/citusdata/pg_shard/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/citusdata/pg_shard/git/refs{/sha}","trees_url":"https://api.github.com/repos/citusdata/pg_shard/git/trees{/sha}","statuses_url":"https://api.github.com/repos/citusdata/pg_shard/statuses/{sha}","languages_url":"https://api.github.com/repos/citusdata/pg_shard/languages","stargazers_url":"https://api.github.com/repos/citusdata/pg_shard/stargazers","contributors_url":"https://api.github.com/repos/citusdata/pg_shard/contributors","subscribers_url":"https://api.github.com/repos/citusdata/pg_shard/subscribers","subscription_url":"https://api.github.com/repos/citusdata/pg_shard/subscription","commits_url":"https://api.github.com/repos/citusdata/pg_shard/commits{/sha}","git_commits_url":"https://api.github.com/repos/citusdata/pg_shard/git/commits{/sha}","comments_url":"https://api.github.com/repos/citusdata/pg_shard/comments{/number}","issue_comment_url":"https://api.github.com/repos/citusdata/pg_shard/issues/comments{/number}","contents_url":"https://api.github.com/repos/citusdata/pg_shard/contents/{+path}","compare_url":"https://api.github.com/repos/citusdata/pg_shard/compare/{base}...{head}","merges_url":"https://api.github.com/repos/citusdata/pg_shard/merges","archive_url":"https://api.github.com/repos/citusdata/pg_shard/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/citusdata/pg_shard/downloads","issues_url":"https://api.github.com/repos/citusdata/pg_shard/issues{/number}","pulls_url":"https://api.github.com/repos/citusdata/pg_shard/pulls{/number}","milestones_url":"https://api.github.com/repos/citusdata/pg_shard/milestones{/number}","notifications_url":"https://api.github.com/repos/citusdata/pg_shard/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/citusdata/pg_shard/labels{/name}","releases_url":"https://api.github.com/repos/citusdata/pg_shard/releases{/id}","created_at":"2014-12-04T04:24:37Z","updated_at":"2015-10-03T00:23:07Z","pushed_at":"2015-10-05T06:51:19Z","git_url":"git://github.com/citusdata/pg_shard.git","ssh_url":"git@github.com:citusdata/pg_shard.git","clone_url":"https://github.com/citusdata/pg_shard.git","svn_url":"https://github.com/citusdata/pg_shard","homepage":"","size":3123,"stargazers_count":894,"watchers_count":894,"language":"C","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":47,"mirror_url":null,"open_issues_count":43,"forks":47,"open_issues":43,"watchers":894,"default_branch":"develop"}},"base":{"label":"citusdata:develop","ref":"develop","sha":"f894c07711684983c805a6187a6015c648dae9af","user":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"repo":{"id":27522888,"name":"pg_shard","full_name":"citusdata/pg_shard","owner":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars.githubusercontent.com/u/2545424?v=3","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/citusdata/pg_shard","description":"PostgreSQL extension to scale out real-time reads and writes. Check out the mailing list at https://groups.google.com/group/pg_shard-users","fork":false,"url":"https://api.github.com/repos/citusdata/pg_shard","forks_url":"https://api.github.com/repos/citusdata/pg_shard/forks","keys_url":"https://api.github.com/repos/citusdata/pg_shard/keys{/key_id}","collaborators_url":"https://api.github.com/repos/citusdata/pg_shard/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/citusdata/pg_shard/teams","hooks_url":"https://api.github.com/repos/citusdata/pg_shard/hooks","issue_events_url":"https://api.github.com/repos/citusdata/pg_shard/issues/events{/number}","events_url":"https://api.github.com/repos/citusdata/pg_shard/events","assignees_url":"https://api.github.com/repos/citusdata/pg_shard/assignees{/user}","branches_url":"https://api.github.com/repos/citusdata/pg_shard/branches{/branch}","tags_url":"https://api.github.com/repos/citusdata/pg_shard/tags","blobs_url":"https://api.github.com/repos/citusdata/pg_shard/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/citusdata/pg_shard/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/citusdata/pg_shard/git/refs{/sha}","trees_url":"https://api.github.com/repos/citusdata/pg_shard/git/trees{/sha}","statuses_url":"https://api.github.com/repos/citusdata/pg_shard/statuses/{sha}","languages_url":"https://api.github.com/repos/citusdata/pg_shard/languages","stargazers_url":"https://api.github.com/repos/citusdata/pg_shard/stargazers","contributors_url":"https://api.github.com/repos/citusdata/pg_shard/contributors","subscribers_url":"https://api.github.com/repos/citusdata/pg_shard/subscribers","subscription_url":"https://api.github.com/repos/citusdata/pg_shard/subscription","commits_url":"https://api.github.com/repos/citusdata/pg_shard/commits{/sha}","git_commits_url":"https://api.github.com/repos/citusdata/pg_shard/git/commits{/sha}","comments_url":"https://api.github.com/repos/citusdata/pg_shard/comments{/number}","issue_comment_url":"https://api.github.com/repos/citusdata/pg_shard/issues/comments{/number}","contents_url":"https://api.github.com/repos/citusdata/pg_shard/contents/{+path}","compare_url":"https://api.github.com/repos/citusdata/pg_shard/compare/{base}...{head}","merges_url":"https://api.github.com/repos/citusdata/pg_shard/merges","archive_url":"https://api.github.com/repos/citusdata/pg_shard/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/citusdata/pg_shard/downloads","issues_url":"https://api.github.com/repos/citusdata/pg_shard/issues{/number}","pulls_url":"https://api.github.com/repos/citusdata/pg_shard/pulls{/number}","milestones_url":"https://api.github.com/repos/citusdata/pg_shard/milestones{/number}","notifications_url":"https://api.github.com/repos/citusdata/pg_shard/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/citusdata/pg_shard/labels{/name}","releases_url":"https://api.github.com/repos/citusdata/pg_shard/releases{/id}","created_at":"2014-12-04T04:24:37Z","updated_at":"2015-10-03T00:23:07Z","pushed_at":"2015-10-05T06:51:19Z","git_url":"git://github.com/citusdata/pg_shard.git","ssh_url":"git@github.com:citusdata/pg_shard.git","clone_url":"https://github.com/citusdata/pg_shard.git","svn_url":"https://github.com/citusdata/pg_shard","homepage":"","size":3123,"stargazers_count":894,"watchers_count":894,"language":"C","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":47,"mirror_url":null,"open_issues_count":43,"forks":47,"open_issues":43,"watchers":894,"default_branch":"develop"}},"_links":{"self":{"href":"https://api.github.com/repos/citusdata/pg_shard/pulls/146"},"html":{"href":"https://github.com/citusdata/pg_shard/pull/146"},"issue":{"href":"https://api.github.com/repos/citusdata/pg_shard/issues/146"},"comments":{"href":"https://api.github.com/repos/citusdata/pg_shard/issues/146/comments"},"review_comments":{"href":"https://api.github.com/repos/citusdata/pg_shard/pulls/146/comments"},"review_comment":{"href":"https://api.github.com/repos/citusdata/pg_shard/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/citusdata/pg_shard/pulls/146/commits"},"statuses":{"href":"https://api.github.com/repos/citusdata/pg_shard/statuses/50578da40b27de03b2d26aa9ed724e518a0dee4d"}},"merged":true,"mergeable":null,"mergeable_state":"unknown","merged_by":{"login":"jasonmp85","id":50673,"avatar_url":"https://avatars.githubusercontent.com/u/50673?v=3","gravatar_id":"","url":"https://api.github.com/users/jasonmp85","html_url":"https://github.com/jasonmp85","followers_url":"https://api.github.com/users/jasonmp85/followers","following_url":"https://api.github.com/users/jasonmp85/following{/other_user}","gists_url":"https://api.github.com/users/jasonmp85/gists{/gist_id}","starred_url":"https://api.github.com/users/jasonmp85/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jasonmp85/subscriptions","organizations_url":"https://api.github.com/users/jasonmp85/orgs","repos_url":"https://api.github.com/users/jasonmp85/repos","events_url":"https://api.github.com/users/jasonmp85/events{/privacy}","received_events_url":"https://api.github.com/users/jasonmp85/received_events","type":"User","site_admin":false},"comments":2,"review_comments":7,"commits":3,"additions":131,"deletions":47,"changed_files":6}}