{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/8825","labels_url":"https://api.github.com/repos/apache/spark/issues/8825/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/8825/comments","events_url":"https://api.github.com/repos/apache/spark/issues/8825/events","html_url":"https://github.com/apache/spark/pull/8825","id":107258548,"number":8825,"title":"[SPARK-10704] Consolidate HashShuffleReader and ShuffleReader; refactor ShuffleManager.getReader()","user":{"login":"JoshRosen","id":50748,"avatar_url":"https://avatars.githubusercontent.com/u/50748?v=3","gravatar_id":"","url":"https://api.github.com/users/JoshRosen","html_url":"https://github.com/JoshRosen","followers_url":"https://api.github.com/users/JoshRosen/followers","following_url":"https://api.github.com/users/JoshRosen/following{/other_user}","gists_url":"https://api.github.com/users/JoshRosen/gists{/gist_id}","starred_url":"https://api.github.com/users/JoshRosen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JoshRosen/subscriptions","organizations_url":"https://api.github.com/users/JoshRosen/orgs","repos_url":"https://api.github.com/users/JoshRosen/repos","events_url":"https://api.github.com/users/JoshRosen/events{/privacy}","received_events_url":"https://api.github.com/users/JoshRosen/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-09-18T19:01:20Z","updated_at":"2015-09-18T21:00:54Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/8825","html_url":"https://github.com/apache/spark/pull/8825","diff_url":"https://github.com/apache/spark/pull/8825.diff","patch_url":"https://github.com/apache/spark/pull/8825.patch"},"body":"The current shuffle code has an interface named ShuffleReader with only one implementation, HashShuffleReader. This naming is confusing, since the same read path code is used for both sort- and hash-based shuffle. This patch consolidates these into a single class named ShuffleReader.\r\n\r\nIn addition, there are aspects of ShuffleManager.getReader()'s API which don't make a lot of sense: it exposes the ability to request a contiguous range of shuffle partitions, but this feature isn't supported by any ShuffleReader implementations and isn't used anywhere in the existing code. If we do want to implement support for fetching multiple partitions then we'll likely want to support the ability to fetch sets of partitions with non-contiguous ids, which this interface doesn't support. As a result, this patch also changes this API to accept a single partition id."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/141567998","html_url":"https://github.com/apache/spark/pull/8825#issuecomment-141567998","issue_url":"https://api.github.com/repos/apache/spark/issues/8825","id":141567998,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2015-09-18T21:00:54Z","updated_at":"2015-09-18T21:00:54Z","body":"  [Test build #42691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42691/consoleFull) for   PR 8825 at commit [`43eb0cf`](https://github.com/apache/spark/commit/43eb0cf2e1ee242724c6f329b8e0ce64b419b806)."}}