{"action":"created","comment":{"url":"https://api.github.com/repos/ceph/ceph/pulls/comments/28409104","id":28409104,"diff_hunk":"@@ -55,7 +55,7 @@ using std::runtime_error;\n #undef dout_prefix\n #define dout_prefix *_dout << \"librados: \"\n \n-#define RADOS_LIST_MAX_ENTRIES 1024\n+#define RADOS_LIST_MAX_ENTRIES 5","path":"src/librados/librados.cc","position":5,"original_position":5,"commit_id":"29e984e64213e7974b9892e576f4a9ccdee6ce0d","original_commit_id":"29e984e64213e7974b9892e576f4a9ccdee6ce0d","user":{"login":"jcsp","id":944640,"avatar_url":"https://avatars.githubusercontent.com/u/944640?v=3","gravatar_id":"","url":"https://api.github.com/users/jcsp","html_url":"https://github.com/jcsp","followers_url":"https://api.github.com/users/jcsp/followers","following_url":"https://api.github.com/users/jcsp/following{/other_user}","gists_url":"https://api.github.com/users/jcsp/gists{/gist_id}","starred_url":"https://api.github.com/users/jcsp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jcsp/subscriptions","organizations_url":"https://api.github.com/users/jcsp/orgs","repos_url":"https://api.github.com/users/jcsp/repos","events_url":"https://api.github.com/users/jcsp/events{/privacy}","received_events_url":"https://api.github.com/users/jcsp/received_events","type":"User","site_admin":false},"body":"Yes, indeed","created_at":"2015-04-15T10:35:52Z","updated_at":"2015-04-15T10:35:52Z","html_url":"https://github.com/ceph/ceph/pull/4360#discussion_r28409104","pull_request_url":"https://api.github.com/repos/ceph/ceph/pulls/4360","_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments/28409104"},"html":{"href":"https://github.com/ceph/ceph/pull/4360#discussion_r28409104"},"pull_request":{"href":"https://api.github.com/repos/ceph/ceph/pulls/4360"}}},"pull_request":{"url":"https://api.github.com/repos/ceph/ceph/pulls/4360","id":33256847,"html_url":"https://github.com/ceph/ceph/pull/4360","diff_url":"https://github.com/ceph/ceph/pull/4360.diff","patch_url":"https://github.com/ceph/ceph/pull/4360.patch","issue_url":"https://api.github.com/repos/ceph/ceph/issues/4360","number":4360,"state":"open","locked":false,"title":"RFC: sharded object listing in librados","user":{"login":"jcsp","id":944640,"avatar_url":"https://avatars.githubusercontent.com/u/944640?v=3","gravatar_id":"","url":"https://api.github.com/users/jcsp","html_url":"https://github.com/jcsp","followers_url":"https://api.github.com/users/jcsp/followers","following_url":"https://api.github.com/users/jcsp/following{/other_user}","gists_url":"https://api.github.com/users/jcsp/gists{/gist_id}","starred_url":"https://api.github.com/users/jcsp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jcsp/subscriptions","organizations_url":"https://api.github.com/users/jcsp/orgs","repos_url":"https://api.github.com/users/jcsp/repos","events_url":"https://api.github.com/users/jcsp/events{/privacy}","received_events_url":"https://api.github.com/users/jcsp/received_events","type":"User","site_admin":false},"body":"This is #9963\r\n\r\nDividing up the object hash space between workers is simple, but then to get a defined fraction of the hash space within a particular PG we have to do some contortions to generate a contiguous range in the nibble-twiddled HashIndex ordering, and make sure that the ranges shared out will get each worker a fair share of objects (as the objects within a given PG are not equally distributed across the hash space).\r\n\r\nI originally played with a different approach that would stripe accesses from many workers across the PGs, but it relied on bitwise masks to assign a PG subset to a worker, and got too awkward when the number of PGs wasn't a power of two.  I ended up with something simpler by assigning each worker a simple range of PGs.\r\n\r\nRFC because I'm still writing the proper tests, and some of the calculation bits are overly verbose with redundant variables for the benefit of my puny meat brain.","created_at":"2015-04-14T13:28:14Z","updated_at":"2015-04-15T10:35:52Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3fe06c3b5b991fc2e5c9f7b7a06637dccf66f516","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/ceph/ceph/pulls/4360/commits","review_comments_url":"https://api.github.com/repos/ceph/ceph/pulls/4360/comments","review_comment_url":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ceph/ceph/issues/4360/comments","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/29e984e64213e7974b9892e576f4a9ccdee6ce0d","head":{"label":"ceph:wip-9963","ref":"wip-9963","sha":"29e984e64213e7974b9892e576f4a9ccdee6ce0d","user":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"repo":{"id":2310495,"name":"ceph","full_name":"ceph/ceph","owner":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ceph/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":false,"url":"https://api.github.com/repos/ceph/ceph","forks_url":"https://api.github.com/repos/ceph/ceph/forks","keys_url":"https://api.github.com/repos/ceph/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ceph/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ceph/ceph/teams","hooks_url":"https://api.github.com/repos/ceph/ceph/hooks","issue_events_url":"https://api.github.com/repos/ceph/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/ceph/ceph/events","assignees_url":"https://api.github.com/repos/ceph/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/ceph/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/ceph/ceph/tags","blobs_url":"https://api.github.com/repos/ceph/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ceph/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ceph/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/ceph/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/ceph/ceph/languages","stargazers_url":"https://api.github.com/repos/ceph/ceph/stargazers","contributors_url":"https://api.github.com/repos/ceph/ceph/contributors","subscribers_url":"https://api.github.com/repos/ceph/ceph/subscribers","subscription_url":"https://api.github.com/repos/ceph/ceph/subscription","commits_url":"https://api.github.com/repos/ceph/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/ceph/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/ceph/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/ceph/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/ceph/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/ceph/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ceph/ceph/merges","archive_url":"https://api.github.com/repos/ceph/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ceph/ceph/downloads","issues_url":"https://api.github.com/repos/ceph/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/ceph/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/ceph/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/ceph/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ceph/ceph/labels{/name}","releases_url":"https://api.github.com/repos/ceph/ceph/releases{/id}","created_at":"2011-09-01T21:41:26Z","updated_at":"2015-04-15T08:30:05Z","pushed_at":"2015-04-15T09:45:52Z","git_url":"git://github.com/ceph/ceph.git","ssh_url":"git@github.com:ceph/ceph.git","clone_url":"https://github.com/ceph/ceph.git","svn_url":"https://github.com/ceph/ceph","homepage":"http://ceph.com","size":404760,"stargazers_count":1241,"watchers_count":1241,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":674,"mirror_url":null,"open_issues_count":126,"forks":674,"open_issues":126,"watchers":1241,"default_branch":"master"}},"base":{"label":"ceph:master","ref":"master","sha":"147c3f2b82d70e46bddd2e3609d7dd6b4faf0b50","user":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"repo":{"id":2310495,"name":"ceph","full_name":"ceph/ceph","owner":{"login":"ceph","id":1015767,"avatar_url":"https://avatars.githubusercontent.com/u/1015767?v=3","gravatar_id":"","url":"https://api.github.com/users/ceph","html_url":"https://github.com/ceph","followers_url":"https://api.github.com/users/ceph/followers","following_url":"https://api.github.com/users/ceph/following{/other_user}","gists_url":"https://api.github.com/users/ceph/gists{/gist_id}","starred_url":"https://api.github.com/users/ceph/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceph/subscriptions","organizations_url":"https://api.github.com/users/ceph/orgs","repos_url":"https://api.github.com/users/ceph/repos","events_url":"https://api.github.com/users/ceph/events{/privacy}","received_events_url":"https://api.github.com/users/ceph/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ceph/ceph","description":"Ceph is a distributed object, block, and file storage platform ","fork":false,"url":"https://api.github.com/repos/ceph/ceph","forks_url":"https://api.github.com/repos/ceph/ceph/forks","keys_url":"https://api.github.com/repos/ceph/ceph/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ceph/ceph/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ceph/ceph/teams","hooks_url":"https://api.github.com/repos/ceph/ceph/hooks","issue_events_url":"https://api.github.com/repos/ceph/ceph/issues/events{/number}","events_url":"https://api.github.com/repos/ceph/ceph/events","assignees_url":"https://api.github.com/repos/ceph/ceph/assignees{/user}","branches_url":"https://api.github.com/repos/ceph/ceph/branches{/branch}","tags_url":"https://api.github.com/repos/ceph/ceph/tags","blobs_url":"https://api.github.com/repos/ceph/ceph/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ceph/ceph/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ceph/ceph/git/refs{/sha}","trees_url":"https://api.github.com/repos/ceph/ceph/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ceph/ceph/statuses/{sha}","languages_url":"https://api.github.com/repos/ceph/ceph/languages","stargazers_url":"https://api.github.com/repos/ceph/ceph/stargazers","contributors_url":"https://api.github.com/repos/ceph/ceph/contributors","subscribers_url":"https://api.github.com/repos/ceph/ceph/subscribers","subscription_url":"https://api.github.com/repos/ceph/ceph/subscription","commits_url":"https://api.github.com/repos/ceph/ceph/commits{/sha}","git_commits_url":"https://api.github.com/repos/ceph/ceph/git/commits{/sha}","comments_url":"https://api.github.com/repos/ceph/ceph/comments{/number}","issue_comment_url":"https://api.github.com/repos/ceph/ceph/issues/comments{/number}","contents_url":"https://api.github.com/repos/ceph/ceph/contents/{+path}","compare_url":"https://api.github.com/repos/ceph/ceph/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ceph/ceph/merges","archive_url":"https://api.github.com/repos/ceph/ceph/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ceph/ceph/downloads","issues_url":"https://api.github.com/repos/ceph/ceph/issues{/number}","pulls_url":"https://api.github.com/repos/ceph/ceph/pulls{/number}","milestones_url":"https://api.github.com/repos/ceph/ceph/milestones{/number}","notifications_url":"https://api.github.com/repos/ceph/ceph/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ceph/ceph/labels{/name}","releases_url":"https://api.github.com/repos/ceph/ceph/releases{/id}","created_at":"2011-09-01T21:41:26Z","updated_at":"2015-04-15T08:30:05Z","pushed_at":"2015-04-15T09:45:52Z","git_url":"git://github.com/ceph/ceph.git","ssh_url":"git@github.com:ceph/ceph.git","clone_url":"https://github.com/ceph/ceph.git","svn_url":"https://github.com/ceph/ceph","homepage":"http://ceph.com","size":404760,"stargazers_count":1241,"watchers_count":1241,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":674,"mirror_url":null,"open_issues_count":126,"forks":674,"open_issues":126,"watchers":1241,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/ceph/ceph/pulls/4360"},"html":{"href":"https://github.com/ceph/ceph/pull/4360"},"issue":{"href":"https://api.github.com/repos/ceph/ceph/issues/4360"},"comments":{"href":"https://api.github.com/repos/ceph/ceph/issues/4360/comments"},"review_comments":{"href":"https://api.github.com/repos/ceph/ceph/pulls/4360/comments"},"review_comment":{"href":"https://api.github.com/repos/ceph/ceph/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ceph/ceph/pulls/4360/commits"},"statuses":{"href":"https://api.github.com/repos/ceph/ceph/statuses/29e984e64213e7974b9892e576f4a9ccdee6ce0d"}}}}