{"action":"created","comment":{"url":"https://api.github.com/repos/amplab/tachyon/pulls/comments/27261164","id":27261164,"diff_hunk":"@@ -115,51 +129,24 @@ public synchronized ClientBlockInfo generateClientBlockInfo(TachyonConf tachyonC\n     ret.blockId = mBlockId;\n     ret.offset = mOffset;\n     ret.length = mLength;\n-    ret.locations = getLocations(tachyonConf);\n-\n-    return ret;\n-  }\n-\n-  /**\n-   * Get the list of pairs \"blockId, workerId\", where the blockId is the id of this block, and the\n-   * workerId is the id of the worker who has the block's data in memory.\n-   *\n-   * @return the list of those pairs\n-   */\n-  public synchronized List<Pair<Long, Long>> getBlockIdWorkerIdPairs() {\n-    List<Pair<Long, Long>> ret = new ArrayList<Pair<Long, Long>>(mLocations.size());\n-    for (long workerId : mLocations.keySet()) {\n-      ret.add(new Pair<Long, Long>(mBlockId, workerId));\n+    ret.workers = new ArrayList<WorkerInfo>();\n+    for (Map.Entry<Long, NetAddress> entry : mLocations.entrySet()) {\n+      List<Long> storageDirIds = new ArrayList<Long>();\n+      storageDirIds.addAll(mWorkerDirs.get(entry.getKey()));\n+      ret.workers.add(new WorkerInfo(entry.getValue(), storageDirIds));\n     }\n+    ret.checkpoints = getCheckpoints(tachyonConf);\n     return ret;\n   }\n \n   /**\n-   * Get the InodeFile of the block\n-   *\n-   * @return the InodeFile of the block\n-   */\n-  public synchronized InodeFile getInodeFile() {\n-    return mInodeFile;\n-  }\n-\n-  /**\n-   * Get the locations of the block, which are the workers' net address who has the data of the\n-   * block in its hierarchy store. The list is sorted by the storage level alias(MEM, SSD, HDD).\n-   * That is, the worker who has the data of the block in its memory is in the top of the list.\n+   * Get the addresses where the block is checkpointed.","path":"core/src/main/java/tachyon/master/BlockInfo.java","position":103,"original_position":103,"commit_id":"e81edc425e6a7eed589ff414a7a961521e464540","original_commit_id":"e81edc425e6a7eed589ff414a7a961521e464540","user":{"login":"calvinjia","id":2223813,"avatar_url":"https://avatars.githubusercontent.com/u/2223813?v=3","gravatar_id":"","url":"https://api.github.com/users/calvinjia","html_url":"https://github.com/calvinjia","followers_url":"https://api.github.com/users/calvinjia/followers","following_url":"https://api.github.com/users/calvinjia/following{/other_user}","gists_url":"https://api.github.com/users/calvinjia/gists{/gist_id}","starred_url":"https://api.github.com/users/calvinjia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/calvinjia/subscriptions","organizations_url":"https://api.github.com/users/calvinjia/orgs","repos_url":"https://api.github.com/users/calvinjia/repos","events_url":"https://api.github.com/users/calvinjia/events{/privacy}","received_events_url":"https://api.github.com/users/calvinjia/received_events","type":"User","site_admin":false},"body":"Should it be hostnames now instead of addresses?","created_at":"2015-03-26T22:08:29Z","updated_at":"2015-03-26T22:08:29Z","html_url":"https://github.com/amplab/tachyon/pull/750#discussion_r27261164","pull_request_url":"https://api.github.com/repos/amplab/tachyon/pulls/750","_links":{"self":{"href":"https://api.github.com/repos/amplab/tachyon/pulls/comments/27261164"},"html":{"href":"https://github.com/amplab/tachyon/pull/750#discussion_r27261164"},"pull_request":{"href":"https://api.github.com/repos/amplab/tachyon/pulls/750"}}},"pull_request":{"url":"https://api.github.com/repos/amplab/tachyon/pulls/750","id":31890428,"html_url":"https://github.com/amplab/tachyon/pull/750","diff_url":"https://github.com/amplab/tachyon/pull/750.diff","patch_url":"https://github.com/amplab/tachyon/pull/750.patch","issue_url":"https://api.github.com/repos/amplab/tachyon/issues/750","number":750,"state":"open","locked":false,"title":"Revamped storageId handling on master and client","user":{"login":"manugoyal","id":628968,"avatar_url":"https://avatars.githubusercontent.com/u/628968?v=3","gravatar_id":"","url":"https://api.github.com/users/manugoyal","html_url":"https://github.com/manugoyal","followers_url":"https://api.github.com/users/manugoyal/followers","following_url":"https://api.github.com/users/manugoyal/following{/other_user}","gists_url":"https://api.github.com/users/manugoyal/gists{/gist_id}","starred_url":"https://api.github.com/users/manugoyal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/manugoyal/subscriptions","organizations_url":"https://api.github.com/users/manugoyal/orgs","repos_url":"https://api.github.com/users/manugoyal/repos","events_url":"https://api.github.com/users/manugoyal/events{/privacy}","received_events_url":"https://api.github.com/users/manugoyal/received_events","type":"User","site_admin":false},"body":"This commit adds storage id information to the thrift metadata, so that\r\nclients can use the information to query workers in the best order. It\r\nalso lets the master handle having multiple storage directories per\r\nworker. While currently we don't store a block on multiple storage\r\ndirectories on a worker, we anticipate this to happen in the future, and\r\nright now it uses minimal overhead.\r\n\r\nThe main changes are to the thrift metadata, where we couple each worker\r\naddress with a list of storage ids. In turn, we modified the BlockInfo\r\nclass to handle this new metadata representation, and we also modified\r\nthe worker storage code to explicitly report removing blocks from a\r\ncertain storageId.\r\n\r\nWe also modified the RemoteBlockInStream class to take advantage of this\r\nnew metadata. Now it will sort the workers by their storage id tier, so\r\nwhen we need to read part of the block remotely, we can query workers in\r\norder of where they have stored the block.","created_at":"2015-03-25T03:23:09Z","updated_at":"2015-03-26T22:08:29Z","closed_at":null,"merged_at":null,"merge_commit_sha":"081df23fd6450bf4ce1d1021e829fd52382556bb","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/amplab/tachyon/pulls/750/commits","review_comments_url":"https://api.github.com/repos/amplab/tachyon/pulls/750/comments","review_comment_url":"https://api.github.com/repos/amplab/tachyon/pulls/comments{/number}","comments_url":"https://api.github.com/repos/amplab/tachyon/issues/750/comments","statuses_url":"https://api.github.com/repos/amplab/tachyon/statuses/e81edc425e6a7eed589ff414a7a961521e464540","head":{"label":"manugoyal:use-storageid","ref":"use-storageid","sha":"e81edc425e6a7eed589ff414a7a961521e464540","user":{"login":"manugoyal","id":628968,"avatar_url":"https://avatars.githubusercontent.com/u/628968?v=3","gravatar_id":"","url":"https://api.github.com/users/manugoyal","html_url":"https://github.com/manugoyal","followers_url":"https://api.github.com/users/manugoyal/followers","following_url":"https://api.github.com/users/manugoyal/following{/other_user}","gists_url":"https://api.github.com/users/manugoyal/gists{/gist_id}","starred_url":"https://api.github.com/users/manugoyal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/manugoyal/subscriptions","organizations_url":"https://api.github.com/users/manugoyal/orgs","repos_url":"https://api.github.com/users/manugoyal/repos","events_url":"https://api.github.com/users/manugoyal/events{/privacy}","received_events_url":"https://api.github.com/users/manugoyal/received_events","type":"User","site_admin":false},"repo":{"id":16339378,"name":"tachyon","full_name":"manugoyal/tachyon","owner":{"login":"manugoyal","id":628968,"avatar_url":"https://avatars.githubusercontent.com/u/628968?v=3","gravatar_id":"","url":"https://api.github.com/users/manugoyal","html_url":"https://github.com/manugoyal","followers_url":"https://api.github.com/users/manugoyal/followers","following_url":"https://api.github.com/users/manugoyal/following{/other_user}","gists_url":"https://api.github.com/users/manugoyal/gists{/gist_id}","starred_url":"https://api.github.com/users/manugoyal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/manugoyal/subscriptions","organizations_url":"https://api.github.com/users/manugoyal/orgs","repos_url":"https://api.github.com/users/manugoyal/repos","events_url":"https://api.github.com/users/manugoyal/events{/privacy}","received_events_url":"https://api.github.com/users/manugoyal/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/manugoyal/tachyon","description":"Reliable File Sharing at Memory Speed Across Cluster Frameworks","fork":true,"url":"https://api.github.com/repos/manugoyal/tachyon","forks_url":"https://api.github.com/repos/manugoyal/tachyon/forks","keys_url":"https://api.github.com/repos/manugoyal/tachyon/keys{/key_id}","collaborators_url":"https://api.github.com/repos/manugoyal/tachyon/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/manugoyal/tachyon/teams","hooks_url":"https://api.github.com/repos/manugoyal/tachyon/hooks","issue_events_url":"https://api.github.com/repos/manugoyal/tachyon/issues/events{/number}","events_url":"https://api.github.com/repos/manugoyal/tachyon/events","assignees_url":"https://api.github.com/repos/manugoyal/tachyon/assignees{/user}","branches_url":"https://api.github.com/repos/manugoyal/tachyon/branches{/branch}","tags_url":"https://api.github.com/repos/manugoyal/tachyon/tags","blobs_url":"https://api.github.com/repos/manugoyal/tachyon/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/manugoyal/tachyon/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/manugoyal/tachyon/git/refs{/sha}","trees_url":"https://api.github.com/repos/manugoyal/tachyon/git/trees{/sha}","statuses_url":"https://api.github.com/repos/manugoyal/tachyon/statuses/{sha}","languages_url":"https://api.github.com/repos/manugoyal/tachyon/languages","stargazers_url":"https://api.github.com/repos/manugoyal/tachyon/stargazers","contributors_url":"https://api.github.com/repos/manugoyal/tachyon/contributors","subscribers_url":"https://api.github.com/repos/manugoyal/tachyon/subscribers","subscription_url":"https://api.github.com/repos/manugoyal/tachyon/subscription","commits_url":"https://api.github.com/repos/manugoyal/tachyon/commits{/sha}","git_commits_url":"https://api.github.com/repos/manugoyal/tachyon/git/commits{/sha}","comments_url":"https://api.github.com/repos/manugoyal/tachyon/comments{/number}","issue_comment_url":"https://api.github.com/repos/manugoyal/tachyon/issues/comments{/number}","contents_url":"https://api.github.com/repos/manugoyal/tachyon/contents/{+path}","compare_url":"https://api.github.com/repos/manugoyal/tachyon/compare/{base}...{head}","merges_url":"https://api.github.com/repos/manugoyal/tachyon/merges","archive_url":"https://api.github.com/repos/manugoyal/tachyon/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/manugoyal/tachyon/downloads","issues_url":"https://api.github.com/repos/manugoyal/tachyon/issues{/number}","pulls_url":"https://api.github.com/repos/manugoyal/tachyon/pulls{/number}","milestones_url":"https://api.github.com/repos/manugoyal/tachyon/milestones{/number}","notifications_url":"https://api.github.com/repos/manugoyal/tachyon/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/manugoyal/tachyon/labels{/name}","releases_url":"https://api.github.com/repos/manugoyal/tachyon/releases{/id}","created_at":"2014-01-29T08:35:14Z","updated_at":"2014-07-17T03:07:23Z","pushed_at":"2015-03-25T03:22:41Z","git_url":"git://github.com/manugoyal/tachyon.git","ssh_url":"git@github.com:manugoyal/tachyon.git","clone_url":"https://github.com/manugoyal/tachyon.git","svn_url":"https://github.com/manugoyal/tachyon","homepage":"http://tachyon-project.org","size":10693,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"amplab:master","ref":"master","sha":"b3edf12f13880845b706b25b73184ca89e9a57bf","user":{"login":"amplab","id":1438454,"avatar_url":"https://avatars.githubusercontent.com/u/1438454?v=3","gravatar_id":"","url":"https://api.github.com/users/amplab","html_url":"https://github.com/amplab","followers_url":"https://api.github.com/users/amplab/followers","following_url":"https://api.github.com/users/amplab/following{/other_user}","gists_url":"https://api.github.com/users/amplab/gists{/gist_id}","starred_url":"https://api.github.com/users/amplab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amplab/subscriptions","organizations_url":"https://api.github.com/users/amplab/orgs","repos_url":"https://api.github.com/users/amplab/repos","events_url":"https://api.github.com/users/amplab/events{/privacy}","received_events_url":"https://api.github.com/users/amplab/received_events","type":"Organization","site_admin":false},"repo":{"id":7276954,"name":"tachyon","full_name":"amplab/tachyon","owner":{"login":"amplab","id":1438454,"avatar_url":"https://avatars.githubusercontent.com/u/1438454?v=3","gravatar_id":"","url":"https://api.github.com/users/amplab","html_url":"https://github.com/amplab","followers_url":"https://api.github.com/users/amplab/followers","following_url":"https://api.github.com/users/amplab/following{/other_user}","gists_url":"https://api.github.com/users/amplab/gists{/gist_id}","starred_url":"https://api.github.com/users/amplab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amplab/subscriptions","organizations_url":"https://api.github.com/users/amplab/orgs","repos_url":"https://api.github.com/users/amplab/repos","events_url":"https://api.github.com/users/amplab/events{/privacy}","received_events_url":"https://api.github.com/users/amplab/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/amplab/tachyon","description":" A Reliable Memory Centric Distributed Storage System","fork":false,"url":"https://api.github.com/repos/amplab/tachyon","forks_url":"https://api.github.com/repos/amplab/tachyon/forks","keys_url":"https://api.github.com/repos/amplab/tachyon/keys{/key_id}","collaborators_url":"https://api.github.com/repos/amplab/tachyon/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/amplab/tachyon/teams","hooks_url":"https://api.github.com/repos/amplab/tachyon/hooks","issue_events_url":"https://api.github.com/repos/amplab/tachyon/issues/events{/number}","events_url":"https://api.github.com/repos/amplab/tachyon/events","assignees_url":"https://api.github.com/repos/amplab/tachyon/assignees{/user}","branches_url":"https://api.github.com/repos/amplab/tachyon/branches{/branch}","tags_url":"https://api.github.com/repos/amplab/tachyon/tags","blobs_url":"https://api.github.com/repos/amplab/tachyon/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/amplab/tachyon/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/amplab/tachyon/git/refs{/sha}","trees_url":"https://api.github.com/repos/amplab/tachyon/git/trees{/sha}","statuses_url":"https://api.github.com/repos/amplab/tachyon/statuses/{sha}","languages_url":"https://api.github.com/repos/amplab/tachyon/languages","stargazers_url":"https://api.github.com/repos/amplab/tachyon/stargazers","contributors_url":"https://api.github.com/repos/amplab/tachyon/contributors","subscribers_url":"https://api.github.com/repos/amplab/tachyon/subscribers","subscription_url":"https://api.github.com/repos/amplab/tachyon/subscription","commits_url":"https://api.github.com/repos/amplab/tachyon/commits{/sha}","git_commits_url":"https://api.github.com/repos/amplab/tachyon/git/commits{/sha}","comments_url":"https://api.github.com/repos/amplab/tachyon/comments{/number}","issue_comment_url":"https://api.github.com/repos/amplab/tachyon/issues/comments{/number}","contents_url":"https://api.github.com/repos/amplab/tachyon/contents/{+path}","compare_url":"https://api.github.com/repos/amplab/tachyon/compare/{base}...{head}","merges_url":"https://api.github.com/repos/amplab/tachyon/merges","archive_url":"https://api.github.com/repos/amplab/tachyon/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/amplab/tachyon/downloads","issues_url":"https://api.github.com/repos/amplab/tachyon/issues{/number}","pulls_url":"https://api.github.com/repos/amplab/tachyon/pulls{/number}","milestones_url":"https://api.github.com/repos/amplab/tachyon/milestones{/number}","notifications_url":"https://api.github.com/repos/amplab/tachyon/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/amplab/tachyon/labels{/name}","releases_url":"https://api.github.com/repos/amplab/tachyon/releases{/id}","created_at":"2012-12-21T17:43:46Z","updated_at":"2015-03-26T17:55:08Z","pushed_at":"2015-03-26T13:41:05Z","git_url":"git://github.com/amplab/tachyon.git","ssh_url":"git@github.com:amplab/tachyon.git","clone_url":"https://github.com/amplab/tachyon.git","svn_url":"https://github.com/amplab/tachyon","homepage":"http://tachyon-project.org","size":26927,"stargazers_count":1254,"watchers_count":1254,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":440,"mirror_url":null,"open_issues_count":18,"forks":440,"open_issues":18,"watchers":1254,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/amplab/tachyon/pulls/750"},"html":{"href":"https://github.com/amplab/tachyon/pull/750"},"issue":{"href":"https://api.github.com/repos/amplab/tachyon/issues/750"},"comments":{"href":"https://api.github.com/repos/amplab/tachyon/issues/750/comments"},"review_comments":{"href":"https://api.github.com/repos/amplab/tachyon/pulls/750/comments"},"review_comment":{"href":"https://api.github.com/repos/amplab/tachyon/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/amplab/tachyon/pulls/750/commits"},"statuses":{"href":"https://api.github.com/repos/amplab/tachyon/statuses/e81edc425e6a7eed589ff414a7a961521e464540"}}}}