{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/6652","labels_url":"https://api.github.com/repos/apache/spark/issues/6652/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/6652/comments","events_url":"https://api.github.com/repos/apache/spark/issues/6652/events","html_url":"https://github.com/apache/spark/pull/6652","id":85258091,"number":6652,"title":"[SPARK-2774] Set preferred locations for reduce tasks","user":{"login":"shivaram","id":143893,"avatar_url":"https://avatars.githubusercontent.com/u/143893?v=3","gravatar_id":"","url":"https://api.github.com/users/shivaram","html_url":"https://github.com/shivaram","followers_url":"https://api.github.com/users/shivaram/followers","following_url":"https://api.github.com/users/shivaram/following{/other_user}","gists_url":"https://api.github.com/users/shivaram/gists{/gist_id}","starred_url":"https://api.github.com/users/shivaram/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shivaram/subscriptions","organizations_url":"https://api.github.com/users/shivaram/orgs","repos_url":"https://api.github.com/users/shivaram/repos","events_url":"https://api.github.com/users/shivaram/events{/privacy}","received_events_url":"https://api.github.com/users/shivaram/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":55,"created_at":"2015-06-04T20:14:38Z","updated_at":"2015-06-09T00:57:17Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/6652","html_url":"https://github.com/apache/spark/pull/6652","diff_url":"https://github.com/apache/spark/pull/6652.diff","patch_url":"https://github.com/apache/spark/pull/6652.patch"},"body":"Set preferred locations for reduce tasks. \r\nThe basic design is that we maintain a map from reducerId to a list of (sizes, locations) for each shuffle. We then pick the machines with the top 5 sizes and set that as preferred locations. \r\n\r\nPicking the top5 needs sorting (well its a little cheaper than sorting, but still), so we restrict this feature to cases where we have fewer than 1000 map tasks and 1000 reduce tasks.\r\n\r\nResubmission of #4576 and  #1697. Hopefully third time is a charm ? cc @kayousterhout for review"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/110185269","html_url":"https://github.com/apache/spark/pull/6652#issuecomment-110185269","issue_url":"https://api.github.com/repos/apache/spark/issues/6652","id":110185269,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-06-09T00:57:17Z","updated_at":"2015-06-09T00:57:17Z","body":"Merged build started."}}