{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/9610","labels_url":"https://api.github.com/repos/apache/spark/issues/9610/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/9610/comments","events_url":"https://api.github.com/repos/apache/spark/issues/9610/events","html_url":"https://github.com/apache/spark/pull/9610","id":116231938,"number":9610,"title":"[SPARK-8029] Robust shuffle writer","user":{"login":"davies","id":40902,"avatar_url":"https://avatars.githubusercontent.com/u/40902?v=3","gravatar_id":"","url":"https://api.github.com/users/davies","html_url":"https://github.com/davies","followers_url":"https://api.github.com/users/davies/followers","following_url":"https://api.github.com/users/davies/following{/other_user}","gists_url":"https://api.github.com/users/davies/gists{/gist_id}","starred_url":"https://api.github.com/users/davies/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davies/subscriptions","organizations_url":"https://api.github.com/users/davies/orgs","repos_url":"https://api.github.com/users/davies/repos","events_url":"https://api.github.com/users/davies/events{/privacy}","received_events_url":"https://api.github.com/users/davies/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":14,"created_at":"2015-11-11T00:10:36Z","updated_at":"2015-11-13T04:22:10Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/9610","html_url":"https://github.com/apache/spark/pull/9610","diff_url":"https://github.com/apache/spark/pull/9610.diff","patch_url":"https://github.com/apache/spark/pull/9610.patch"},"body":"Currently, all the shuffle writer will write to target path directly, the file could be corrupted by other attempt of the same partition on the same executor. They should write to temporary file then rename to target path, as what we do in output committer. In order to make the rename atomic, the temporary file should be created in the same local directory (FileSystem).\r\n\r\nThis PR is based on #9214 , thanks to @squito . Closes #9214"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/156318305","html_url":"https://github.com/apache/spark/pull/9610#issuecomment-156318305","issue_url":"https://api.github.com/repos/apache/spark/issues/9610","id":156318305,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-11-13T04:22:10Z","updated_at":"2015-11-13T04:22:10Z","body":" Merged build triggered."}}