{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/10269","labels_url":"https://api.github.com/repos/apache/spark/issues/10269/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/10269/comments","events_url":"https://api.github.com/repos/apache/spark/issues/10269/events","html_url":"https://github.com/apache/spark/pull/10269","id":121798101,"number":10269,"title":"[SPARK-12281][Core]Fix a race condition when reporting ExecutorState in the shutdown hook","user":{"login":"zsxwing","id":1000778,"avatar_url":"https://avatars.githubusercontent.com/u/1000778?v=3","gravatar_id":"","url":"https://api.github.com/users/zsxwing","html_url":"https://github.com/zsxwing","followers_url":"https://api.github.com/users/zsxwing/followers","following_url":"https://api.github.com/users/zsxwing/following{/other_user}","gists_url":"https://api.github.com/users/zsxwing/gists{/gist_id}","starred_url":"https://api.github.com/users/zsxwing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zsxwing/subscriptions","organizations_url":"https://api.github.com/users/zsxwing/orgs","repos_url":"https://api.github.com/users/zsxwing/repos","events_url":"https://api.github.com/users/zsxwing/events{/privacy}","received_events_url":"https://api.github.com/users/zsxwing/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-12-11T21:48:43Z","updated_at":"2015-12-12T00:08:52Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/10269","html_url":"https://github.com/apache/spark/pull/10269","diff_url":"https://github.com/apache/spark/pull/10269.diff","patch_url":"https://github.com/apache/spark/pull/10269.patch"},"body":"1. Make sure workers and masters exit so that no worker or master will still be running when triggering the shutdown hook.\r\n2. Set ExecutorState to FAILED if it's still RUNNING when executing the shutdown hook.\r\n\r\nThis should fix the potential exceptions when exiting a local cluster\r\n```\r\njava.lang.AssertionError: assertion failed: executor 4 state transfer from RUNNING to RUNNING is illegal\r\n\tat scala.Predef$.assert(Predef.scala:179)\r\n\tat org.apache.spark.deploy.master.Master$$anonfun$receive$1.applyOrElse(Master.scala:260)\r\n\tat org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:116)\r\n\tat org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:204)\r\n\tat org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)\r\n\tat org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:215)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n\r\njava.lang.IllegalStateException: Shutdown hooks cannot be modified during shutdown.\r\n\tat org.apache.spark.util.SparkShutdownHookManager.add(ShutdownHookManager.scala:246)\r\n\tat org.apache.spark.util.ShutdownHookManager$.addShutdownHook(ShutdownHookManager.scala:191)\r\n\tat org.apache.spark.util.ShutdownHookManager$.addShutdownHook(ShutdownHookManager.scala:180)\r\n\tat org.apache.spark.deploy.worker.ExecutorRunner.start(ExecutorRunner.scala:73)\r\n\tat org.apache.spark.deploy.worker.Worker$$anonfun$receive$1.applyOrElse(Worker.scala:474)\r\n\tat org.apache.spark.rpc.netty.Inbox$$anonfun$process$1.apply$mcV$sp(Inbox.scala:116)\r\n\tat org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:204)\r\n\tat org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)\r\n\tat org.apache.spark.rpc.netty.Dispatcher$MessageLoop.run(Dispatcher.scala:215)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n```"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/164085553","html_url":"https://github.com/apache/spark/pull/10269#issuecomment-164085553","issue_url":"https://api.github.com/repos/apache/spark/issues/10269","id":164085553,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2015-12-12T00:08:51Z","updated_at":"2015-12-12T00:08:51Z","body":"**[Test build #47593 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47593/consoleFull)** for PR 10269 at commit [`bedaead`](https://github.com/apache/spark/commit/bedaeade3ae7621452acb7c59391afbbb9cb3ecd).\n * This patch passes all tests.\n * This patch merges cleanly.\n * This patch adds the following public classes _(experimental)_:\\n  * `class ExecutorClassLoader(`\\n"}}