{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/3638","labels_url":"https://api.github.com/repos/apache/spark/issues/3638/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/3638/comments","events_url":"https://api.github.com/repos/apache/spark/issues/3638/events","html_url":"https://github.com/apache/spark/pull/3638","id":51349729,"number":3638,"title":"[SPARK-4737] Task set manager properly handles serialization errors","user":{"login":"mccheah","id":8975385,"avatar_url":"https://avatars.githubusercontent.com/u/8975385?v=3","gravatar_id":"","url":"https://api.github.com/users/mccheah","html_url":"https://github.com/mccheah","followers_url":"https://api.github.com/users/mccheah/followers","following_url":"https://api.github.com/users/mccheah/following{/other_user}","gists_url":"https://api.github.com/users/mccheah/gists{/gist_id}","starred_url":"https://api.github.com/users/mccheah/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mccheah/subscriptions","organizations_url":"https://api.github.com/users/mccheah/orgs","repos_url":"https://api.github.com/users/mccheah/repos","events_url":"https://api.github.com/users/mccheah/events{/privacy}","received_events_url":"https://api.github.com/users/mccheah/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":35,"created_at":"2014-12-08T20:44:14Z","updated_at":"2015-01-09T03:44:37Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/3638","html_url":"https://github.com/apache/spark/pull/3638","diff_url":"https://github.com/apache/spark/pull/3638.diff","patch_url":"https://github.com/apache/spark/pull/3638.patch"},"body":"Dealing with [SPARK-4737], the handling of serialization errors should not be the DAGScheduler's responsibility. The task set manager now catches the error and aborts the stage.\r\n\r\nIf the TaskSetManager throws a TaskNotSerializableException, the TaskSchedulerImpl will return an empty list of task descriptions, because no tasks were started. The scheduler should abort the stage gracefully.\r\n\r\nNote that I'm not too familiar with this part of the codebase and its place in the overall architecture of the Spark stack. If implementing it this way will have any averse side effects please voice that loudly."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/69289517","html_url":"https://github.com/apache/spark/pull/3638#issuecomment-69289517","issue_url":"https://api.github.com/repos/apache/spark/issues/3638","id":69289517,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2015-01-09T03:44:37Z","updated_at":"2015-01-09T03:44:37Z","body":"  [Test build #25289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25289/consoleFull) for   PR 3638 at commit [`1545984`](https://github.com/apache/spark/commit/154598404f181fbb761f38385b2e844c792c1b03).\n * This patch **passes all tests**.\n * This patch merges cleanly.\n * This patch adds no public classes."}}