{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/9227","labels_url":"https://api.github.com/repos/apache/spark/issues/9227/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/9227/comments","events_url":"https://api.github.com/repos/apache/spark/issues/9227/events","html_url":"https://github.com/apache/spark/pull/9227","id":112838801,"number":9227,"title":"[spark-11252][network]ShuffleClient should release connection after fetching blocks had been completed for external shuffle","user":{"login":"lianhuiwang","id":545478,"avatar_url":"https://avatars.githubusercontent.com/u/545478?v=3","gravatar_id":"","url":"https://api.github.com/users/lianhuiwang","html_url":"https://github.com/lianhuiwang","followers_url":"https://api.github.com/users/lianhuiwang/followers","following_url":"https://api.github.com/users/lianhuiwang/following{/other_user}","gists_url":"https://api.github.com/users/lianhuiwang/gists{/gist_id}","starred_url":"https://api.github.com/users/lianhuiwang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lianhuiwang/subscriptions","organizations_url":"https://api.github.com/users/lianhuiwang/orgs","repos_url":"https://api.github.com/users/lianhuiwang/repos","events_url":"https://api.github.com/users/lianhuiwang/events{/privacy}","received_events_url":"https://api.github.com/users/lianhuiwang/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":14,"created_at":"2015-10-22T16:16:03Z","updated_at":"2015-11-10T00:48:05Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/9227","html_url":"https://github.com/apache/spark/pull/9227","diff_url":"https://github.com/apache/spark/pull/9227.diff","patch_url":"https://github.com/apache/spark/pull/9227.patch"},"body":"with yarn's external shuffle, ExternalShuffleClient of executors reserve its connections for yarn's NodeManager until application has been completed. so it will make NodeManager and executors have many socket connections.\r\nin order to reduce network pressure of NodeManager's shuffleService, after registerWithShuffleServer or fetchBlocks have been completed in ExternalShuffleClient, connection for NM's shuffleService needs to be closed.@andrewor14 @rxin @vanzin "},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/155247552","html_url":"https://github.com/apache/spark/pull/9227#issuecomment-155247552","issue_url":"https://api.github.com/repos/apache/spark/issues/9227","id":155247552,"user":{"login":"vanzin","id":1694083,"avatar_url":"https://avatars.githubusercontent.com/u/1694083?v=3","gravatar_id":"","url":"https://api.github.com/users/vanzin","html_url":"https://github.com/vanzin","followers_url":"https://api.github.com/users/vanzin/followers","following_url":"https://api.github.com/users/vanzin/following{/other_user}","gists_url":"https://api.github.com/users/vanzin/gists{/gist_id}","starred_url":"https://api.github.com/users/vanzin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vanzin/subscriptions","organizations_url":"https://api.github.com/users/vanzin/orgs","repos_url":"https://api.github.com/users/vanzin/repos","events_url":"https://api.github.com/users/vanzin/events{/privacy}","received_events_url":"https://api.github.com/users/vanzin/received_events","type":"User","site_admin":false},"created_at":"2015-11-10T00:48:05Z","updated_at":"2015-11-10T00:48:05Z","body":"A couple of small things left, otherwise LGTM."}}