{"action":"created","comment":{"url":"https://api.github.com/repos/baidu/tera/pulls/comments/40630254","id":40630254,"diff_hunk":"@@ -845,7 +874,8 @@ void DBImpl::BackgroundCall() {\n     }\n   }\n \n-  bg_compaction_scheduled_ = false;\n+  //bg_compaction_scheduled_ = false;\n+  bg_compaction_scheduled_count_--;","path":"src/leveldb/db/db_impl.cc","position":114,"original_position":114,"commit_id":"7e4487c88882e297fe8d701cd79fc9fd88acce03","original_commit_id":"7e4487c88882e297fe8d701cd79fc9fd88acce03","user":{"login":"taocp","id":5194281,"avatar_url":"https://avatars.githubusercontent.com/u/5194281?v=3","gravatar_id":"","url":"https://api.github.com/users/taocp","html_url":"https://github.com/taocp","followers_url":"https://api.github.com/users/taocp/followers","following_url":"https://api.github.com/users/taocp/following{/other_user}","gists_url":"https://api.github.com/users/taocp/gists{/gist_id}","starred_url":"https://api.github.com/users/taocp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/taocp/subscriptions","organizations_url":"https://api.github.com/users/taocp/orgs","repos_url":"https://api.github.com/users/taocp/repos","events_url":"https://api.github.com/users/taocp/events{/privacy}","received_events_url":"https://api.github.com/users/taocp/received_events","type":"User","site_admin":false},"body":"关于`MaybeScheduleCompaction()`和`SchedulePendingCompaction()`有些问题，等放假回来当面讨论。","created_at":"2015-09-29T02:52:29Z","updated_at":"2015-09-29T02:52:29Z","html_url":"https://github.com/baidu/tera/pull/335#discussion_r40630254","pull_request_url":"https://api.github.com/repos/baidu/tera/pulls/335","_links":{"self":{"href":"https://api.github.com/repos/baidu/tera/pulls/comments/40630254"},"html":{"href":"https://github.com/baidu/tera/pull/335#discussion_r40630254"},"pull_request":{"href":"https://api.github.com/repos/baidu/tera/pulls/335"}}},"pull_request":{"url":"https://api.github.com/repos/baidu/tera/pulls/335","id":45508941,"html_url":"https://github.com/baidu/tera/pull/335","diff_url":"https://github.com/baidu/tera/pull/335.diff","patch_url":"https://github.com/baidu/tera/pull/335.patch","issue_url":"https://api.github.com/repos/baidu/tera/issues/335","number":335,"state":"open","locked":false,"title":"issue=#98 Multi thread compaction","user":{"login":"taocp","id":5194281,"avatar_url":"https://avatars.githubusercontent.com/u/5194281?v=3","gravatar_id":"","url":"https://api.github.com/users/taocp","html_url":"https://github.com/taocp","followers_url":"https://api.github.com/users/taocp/followers","following_url":"https://api.github.com/users/taocp/following{/other_user}","gists_url":"https://api.github.com/users/taocp/gists{/gist_id}","starred_url":"https://api.github.com/users/taocp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/taocp/subscriptions","organizations_url":"https://api.github.com/users/taocp/orgs","repos_url":"https://api.github.com/users/taocp/repos","events_url":"https://api.github.com/users/taocp/events{/privacy}","received_events_url":"https://api.github.com/users/taocp/received_events","type":"User","site_admin":false},"body":"朴素的多线程compact实现，主要参考了rocksdb的代码。\r\n请先review下实现的思路，细节方面还有待改进（诸如日志等）。\r\n\r\n实现的思路：\r\n每个sst文件都带上标志位，表示该sst目前是否参与compact.\r\n带锁去执行pick，保证pick出来的sst文件没有处于compact状态，并发地pick出来多个不冲突的compaction任务，并发执行compact。\r\n\r\npick的具体策略：\r\n1. 先计算每个level的分数，计算时处于compact状态的sst不影响该level的分数，其它策略和原有的分数计算一样；\r\n2. 再从最高分的level开始找，试图找到一个分数>=1.0 且可以 compact 的 level；\r\n3. 对于某个level是否可以compact，执行如下操作：\r\n  * 首先，遍历该level的所有sst，找到一个**自身**没有参与compact的sst\r\n  * 查看这个sst的range映射到parent-level时，**parent-level对应的sst**是否处于compact状态\r\n  * 如果以上两个都没有处于compact状态，则这个 sst 以及它对应的parent sst可以compact\r\n\r\n因为level0有点特殊，目前不支持并发compact，其实对于level0，可以根据range来并发（其它level因为level内部不会有重叠，所以可以根据sst来并发），可以进一步优化。","created_at":"2015-09-21T01:25:47Z","updated_at":"2015-09-29T02:52:29Z","closed_at":null,"merged_at":null,"merge_commit_sha":"9a736f4b19cde3468718b41b910113c6fcefaa20","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/baidu/tera/pulls/335/commits","review_comments_url":"https://api.github.com/repos/baidu/tera/pulls/335/comments","review_comment_url":"https://api.github.com/repos/baidu/tera/pulls/comments{/number}","comments_url":"https://api.github.com/repos/baidu/tera/issues/335/comments","statuses_url":"https://api.github.com/repos/baidu/tera/statuses/7e4487c88882e297fe8d701cd79fc9fd88acce03","head":{"label":"taocp:multi-thread-compaction-diff","ref":"multi-thread-compaction-diff","sha":"7e4487c88882e297fe8d701cd79fc9fd88acce03","user":{"login":"taocp","id":5194281,"avatar_url":"https://avatars.githubusercontent.com/u/5194281?v=3","gravatar_id":"","url":"https://api.github.com/users/taocp","html_url":"https://github.com/taocp","followers_url":"https://api.github.com/users/taocp/followers","following_url":"https://api.github.com/users/taocp/following{/other_user}","gists_url":"https://api.github.com/users/taocp/gists{/gist_id}","starred_url":"https://api.github.com/users/taocp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/taocp/subscriptions","organizations_url":"https://api.github.com/users/taocp/orgs","repos_url":"https://api.github.com/users/taocp/repos","events_url":"https://api.github.com/users/taocp/events{/privacy}","received_events_url":"https://api.github.com/users/taocp/received_events","type":"User","site_admin":false},"repo":{"id":32839075,"name":"tera","full_name":"taocp/tera","owner":{"login":"taocp","id":5194281,"avatar_url":"https://avatars.githubusercontent.com/u/5194281?v=3","gravatar_id":"","url":"https://api.github.com/users/taocp","html_url":"https://github.com/taocp","followers_url":"https://api.github.com/users/taocp/followers","following_url":"https://api.github.com/users/taocp/following{/other_user}","gists_url":"https://api.github.com/users/taocp/gists{/gist_id}","starred_url":"https://api.github.com/users/taocp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/taocp/subscriptions","organizations_url":"https://api.github.com/users/taocp/orgs","repos_url":"https://api.github.com/users/taocp/repos","events_url":"https://api.github.com/users/taocp/events{/privacy}","received_events_url":"https://api.github.com/users/taocp/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/taocp/tera","description":"A distributed table storage system.","fork":true,"url":"https://api.github.com/repos/taocp/tera","forks_url":"https://api.github.com/repos/taocp/tera/forks","keys_url":"https://api.github.com/repos/taocp/tera/keys{/key_id}","collaborators_url":"https://api.github.com/repos/taocp/tera/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/taocp/tera/teams","hooks_url":"https://api.github.com/repos/taocp/tera/hooks","issue_events_url":"https://api.github.com/repos/taocp/tera/issues/events{/number}","events_url":"https://api.github.com/repos/taocp/tera/events","assignees_url":"https://api.github.com/repos/taocp/tera/assignees{/user}","branches_url":"https://api.github.com/repos/taocp/tera/branches{/branch}","tags_url":"https://api.github.com/repos/taocp/tera/tags","blobs_url":"https://api.github.com/repos/taocp/tera/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/taocp/tera/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/taocp/tera/git/refs{/sha}","trees_url":"https://api.github.com/repos/taocp/tera/git/trees{/sha}","statuses_url":"https://api.github.com/repos/taocp/tera/statuses/{sha}","languages_url":"https://api.github.com/repos/taocp/tera/languages","stargazers_url":"https://api.github.com/repos/taocp/tera/stargazers","contributors_url":"https://api.github.com/repos/taocp/tera/contributors","subscribers_url":"https://api.github.com/repos/taocp/tera/subscribers","subscription_url":"https://api.github.com/repos/taocp/tera/subscription","commits_url":"https://api.github.com/repos/taocp/tera/commits{/sha}","git_commits_url":"https://api.github.com/repos/taocp/tera/git/commits{/sha}","comments_url":"https://api.github.com/repos/taocp/tera/comments{/number}","issue_comment_url":"https://api.github.com/repos/taocp/tera/issues/comments{/number}","contents_url":"https://api.github.com/repos/taocp/tera/contents/{+path}","compare_url":"https://api.github.com/repos/taocp/tera/compare/{base}...{head}","merges_url":"https://api.github.com/repos/taocp/tera/merges","archive_url":"https://api.github.com/repos/taocp/tera/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/taocp/tera/downloads","issues_url":"https://api.github.com/repos/taocp/tera/issues{/number}","pulls_url":"https://api.github.com/repos/taocp/tera/pulls{/number}","milestones_url":"https://api.github.com/repos/taocp/tera/milestones{/number}","notifications_url":"https://api.github.com/repos/taocp/tera/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/taocp/tera/labels{/name}","releases_url":"https://api.github.com/repos/taocp/tera/releases{/id}","created_at":"2015-03-25T02:57:20Z","updated_at":"2015-05-05T11:37:02Z","pushed_at":"2015-09-28T02:06:02Z","git_url":"git://github.com/taocp/tera.git","ssh_url":"git@github.com:taocp/tera.git","clone_url":"https://github.com/taocp/tera.git","svn_url":"https://github.com/taocp/tera","homepage":"","size":12336,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":3,"forks":0,"open_issues":3,"watchers":0,"default_branch":"master"}},"base":{"label":"baidu:master","ref":"master","sha":"54b092a1d0dab505c4233d51333c2a00b0e83dec","user":{"login":"baidu","id":13245940,"avatar_url":"https://avatars.githubusercontent.com/u/13245940?v=3","gravatar_id":"","url":"https://api.github.com/users/baidu","html_url":"https://github.com/baidu","followers_url":"https://api.github.com/users/baidu/followers","following_url":"https://api.github.com/users/baidu/following{/other_user}","gists_url":"https://api.github.com/users/baidu/gists{/gist_id}","starred_url":"https://api.github.com/users/baidu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/baidu/subscriptions","organizations_url":"https://api.github.com/users/baidu/orgs","repos_url":"https://api.github.com/users/baidu/repos","events_url":"https://api.github.com/users/baidu/events{/privacy}","received_events_url":"https://api.github.com/users/baidu/received_events","type":"Organization","site_admin":false},"repo":{"id":18129915,"name":"tera","full_name":"baidu/tera","owner":{"login":"baidu","id":13245940,"avatar_url":"https://avatars.githubusercontent.com/u/13245940?v=3","gravatar_id":"","url":"https://api.github.com/users/baidu","html_url":"https://github.com/baidu","followers_url":"https://api.github.com/users/baidu/followers","following_url":"https://api.github.com/users/baidu/following{/other_user}","gists_url":"https://api.github.com/users/baidu/gists{/gist_id}","starred_url":"https://api.github.com/users/baidu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/baidu/subscriptions","organizations_url":"https://api.github.com/users/baidu/orgs","repos_url":"https://api.github.com/users/baidu/repos","events_url":"https://api.github.com/users/baidu/events{/privacy}","received_events_url":"https://api.github.com/users/baidu/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/baidu/tera","description":"A distributed, elastic, structured database.","fork":false,"url":"https://api.github.com/repos/baidu/tera","forks_url":"https://api.github.com/repos/baidu/tera/forks","keys_url":"https://api.github.com/repos/baidu/tera/keys{/key_id}","collaborators_url":"https://api.github.com/repos/baidu/tera/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/baidu/tera/teams","hooks_url":"https://api.github.com/repos/baidu/tera/hooks","issue_events_url":"https://api.github.com/repos/baidu/tera/issues/events{/number}","events_url":"https://api.github.com/repos/baidu/tera/events","assignees_url":"https://api.github.com/repos/baidu/tera/assignees{/user}","branches_url":"https://api.github.com/repos/baidu/tera/branches{/branch}","tags_url":"https://api.github.com/repos/baidu/tera/tags","blobs_url":"https://api.github.com/repos/baidu/tera/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/baidu/tera/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/baidu/tera/git/refs{/sha}","trees_url":"https://api.github.com/repos/baidu/tera/git/trees{/sha}","statuses_url":"https://api.github.com/repos/baidu/tera/statuses/{sha}","languages_url":"https://api.github.com/repos/baidu/tera/languages","stargazers_url":"https://api.github.com/repos/baidu/tera/stargazers","contributors_url":"https://api.github.com/repos/baidu/tera/contributors","subscribers_url":"https://api.github.com/repos/baidu/tera/subscribers","subscription_url":"https://api.github.com/repos/baidu/tera/subscription","commits_url":"https://api.github.com/repos/baidu/tera/commits{/sha}","git_commits_url":"https://api.github.com/repos/baidu/tera/git/commits{/sha}","comments_url":"https://api.github.com/repos/baidu/tera/comments{/number}","issue_comment_url":"https://api.github.com/repos/baidu/tera/issues/comments{/number}","contents_url":"https://api.github.com/repos/baidu/tera/contents/{+path}","compare_url":"https://api.github.com/repos/baidu/tera/compare/{base}...{head}","merges_url":"https://api.github.com/repos/baidu/tera/merges","archive_url":"https://api.github.com/repos/baidu/tera/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/baidu/tera/downloads","issues_url":"https://api.github.com/repos/baidu/tera/issues{/number}","pulls_url":"https://api.github.com/repos/baidu/tera/pulls{/number}","milestones_url":"https://api.github.com/repos/baidu/tera/milestones{/number}","notifications_url":"https://api.github.com/repos/baidu/tera/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/baidu/tera/labels{/name}","releases_url":"https://api.github.com/repos/baidu/tera/releases{/id}","created_at":"2014-03-26T07:22:05Z","updated_at":"2015-09-28T11:41:55Z","pushed_at":"2015-09-29T01:44:51Z","git_url":"git://github.com/baidu/tera.git","ssh_url":"git@github.com:baidu/tera.git","clone_url":"https://github.com/baidu/tera.git","svn_url":"https://github.com/baidu/tera","homepage":"","size":16924,"stargazers_count":287,"watchers_count":287,"language":"C++","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":90,"mirror_url":null,"open_issues_count":48,"forks":90,"open_issues":48,"watchers":287,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/baidu/tera/pulls/335"},"html":{"href":"https://github.com/baidu/tera/pull/335"},"issue":{"href":"https://api.github.com/repos/baidu/tera/issues/335"},"comments":{"href":"https://api.github.com/repos/baidu/tera/issues/335/comments"},"review_comments":{"href":"https://api.github.com/repos/baidu/tera/pulls/335/comments"},"review_comment":{"href":"https://api.github.com/repos/baidu/tera/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/baidu/tera/pulls/335/commits"},"statuses":{"href":"https://api.github.com/repos/baidu/tera/statuses/7e4487c88882e297fe8d701cd79fc9fd88acce03"}}}}