{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/6833","labels_url":"https://api.github.com/repos/apache/spark/issues/6833/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/6833/comments","events_url":"https://api.github.com/repos/apache/spark/issues/6833/events","html_url":"https://github.com/apache/spark/pull/6833","id":88589613,"number":6833,"title":"[SPARK-8379][SQL]avoid speculative tasks write to the same file","user":{"login":"jeanlyn","id":3426093,"avatar_url":"https://avatars.githubusercontent.com/u/3426093?v=3","gravatar_id":"","url":"https://api.github.com/users/jeanlyn","html_url":"https://github.com/jeanlyn","followers_url":"https://api.github.com/users/jeanlyn/followers","following_url":"https://api.github.com/users/jeanlyn/following{/other_user}","gists_url":"https://api.github.com/users/jeanlyn/gists{/gist_id}","starred_url":"https://api.github.com/users/jeanlyn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeanlyn/subscriptions","organizations_url":"https://api.github.com/users/jeanlyn/orgs","repos_url":"https://api.github.com/users/jeanlyn/repos","events_url":"https://api.github.com/users/jeanlyn/events{/privacy}","received_events_url":"https://api.github.com/users/jeanlyn/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-06-16T02:00:54Z","updated_at":"2015-06-17T01:39:30Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/6833","html_url":"https://github.com/apache/spark/pull/6833","diff_url":"https://github.com/apache/spark/pull/6833.diff","patch_url":"https://github.com/apache/spark/pull/6833.patch"},"body":"The issue link [SPARK-8379](https://issues.apache.org/jira/browse/SPARK-8379)\r\nCurrently,when we insert data to the dynamic partition with speculative tasks we will get the Exception\r\n```\r\norg.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException): \r\nLease mismatch on /tmp/hive-jeanlyn/hive_2015-06-15_15-20-44_734_8801220787219172413-1/-ext-10000/ds=2015-06-15/type=2/part-00301.lzo \r\nowned by DFSClient_attempt_201506031520_0011_m_000189_0_-1513487243_53 \r\nbut is accessed by DFSClient_attempt_201506031520_0011_m_000042_0_-1275047721_57\r\n```\r\nThis pr try to write the data to temporary dir when using dynamic parition  avoid the speculative tasks writing the same file"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/112620496","html_url":"https://github.com/apache/spark/pull/6833#issuecomment-112620496","issue_url":"https://api.github.com/repos/apache/spark/issues/6833","id":112620496,"user":{"login":"chenghao-intel","id":2367031,"avatar_url":"https://avatars.githubusercontent.com/u/2367031?v=3","gravatar_id":"","url":"https://api.github.com/users/chenghao-intel","html_url":"https://github.com/chenghao-intel","followers_url":"https://api.github.com/users/chenghao-intel/followers","following_url":"https://api.github.com/users/chenghao-intel/following{/other_user}","gists_url":"https://api.github.com/users/chenghao-intel/gists{/gist_id}","starred_url":"https://api.github.com/users/chenghao-intel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chenghao-intel/subscriptions","organizations_url":"https://api.github.com/users/chenghao-intel/orgs","repos_url":"https://api.github.com/users/chenghao-intel/repos","events_url":"https://api.github.com/users/chenghao-intel/events{/privacy}","received_events_url":"https://api.github.com/users/chenghao-intel/received_events","type":"User","site_admin":false},"created_at":"2015-06-17T01:39:30Z","updated_at":"2015-06-17T01:39:30Z","body":"Seems only affect the dynamic partition in HiveContext, @jeanlyn can you confirm that?"}}