{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/9684","labels_url":"https://api.github.com/repos/apache/spark/issues/9684/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/9684/comments","events_url":"https://api.github.com/repos/apache/spark/issues/9684/events","html_url":"https://github.com/apache/spark/pull/9684","id":116705811,"number":9684,"title":"[SPARK-11718][Yarn][Core]Fix explicitly killing executor dies silently issue","user":{"login":"jerryshao","id":850797,"avatar_url":"https://avatars.githubusercontent.com/u/850797?v=3","gravatar_id":"","url":"https://api.github.com/users/jerryshao","html_url":"https://github.com/jerryshao","followers_url":"https://api.github.com/users/jerryshao/followers","following_url":"https://api.github.com/users/jerryshao/following{/other_user}","gists_url":"https://api.github.com/users/jerryshao/gists{/gist_id}","starred_url":"https://api.github.com/users/jerryshao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jerryshao/subscriptions","organizations_url":"https://api.github.com/users/jerryshao/orgs","repos_url":"https://api.github.com/users/jerryshao/repos","events_url":"https://api.github.com/users/jerryshao/events{/privacy}","received_events_url":"https://api.github.com/users/jerryshao/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-11-13T05:59:23Z","updated_at":"2015-11-13T06:02:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/9684","html_url":"https://github.com/apache/spark/pull/9684","diff_url":"https://github.com/apache/spark/pull/9684.diff","patch_url":"https://github.com/apache/spark/pull/9684.patch"},"body":"Currently if dynamic allocation is enabled, explicitly killing executor will not get response, so the executor metadata is wrong in driver side. Which will make dynamic allocation on Yarn fail to work.\r\n\r\nThe problem is  `disableExecutor` returns false for pending killing executors when `onDisconnect` is detected, so no further implementation is done.\r\n\r\nOne solution is to bypass these explicitly killed executors to use `super.onDisconnect` to remove executor. This is simple.\r\n\r\nAnother solution is still querying the loss reason for these explicitly kill executors. Since executor may get killed and informed in the same AM-RM communication, so current way of adding pending loss reason request is not worked (container complete is already processed), here we should store this loss reason for later query.\r\n\r\nHere this PR chooses solution 2.\r\n\r\nPlease help to review. @vanzin I think this part is changed by you previously, would you please help to review? Thanks a lot."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/156332450","html_url":"https://github.com/apache/spark/pull/9684#issuecomment-156332450","issue_url":"https://api.github.com/repos/apache/spark/issues/9684","id":156332450,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-11-13T06:02:18Z","updated_at":"2015-11-13T06:02:18Z","body":"Merged build started."}}