{"push_id":879851168,"size":2,"distinct_size":2,"ref":"refs/heads/branch-1.5","head":"b1fcefca6b257f5bb1be85fce2b4a97d4271e0aa","before":"27b5f31a08737969b69a43536acc04cb882a7bdc","commits":[{"sha":"4139a4ed1f024b1a88e4efe613acf45175f82318","author":{"email":"47f947204c0a430d1f3d77ee35599c230fda5709@gmail.com","name":"Mark Hamstra"},"message":"[SPARK-10666][SPARK-6880][CORE] Use properties from ActiveJob associated with a Stage\n\nThis issue was addressed in https://github.com/apache/spark/pull/5494, but the fix in that PR, while safe in the sense that it will prevent the SparkContext from shutting down, misses the actual bug.  The intent of `submitMissingTasks` should be understood as \"submit the Tasks that are missing for the Stage, and run them as part of the ActiveJob identified by jobId\".  Because of a long-standing bug, the `jobId` parameter was never being used.  Instead, we were trying to use the jobId with which the Stage was created -- which may no longer exist as an ActiveJob, hence the crash reported in SPARK-6880.\n\nThe correct fix is to use the ActiveJob specified by the supplied jobId parameter, which is guaranteed to exist at the call sites of submitMissingTasks.\n\nThis fix should be applied to all maintenance branches, since it has existed since 1.0.\n\nkayousterhout pankajarora12\n\nAuthor: Mark Hamstra <markhamstra@gmail.com>\nAuthor: Imran Rashid <irashid@cloudera.com>\n\nCloses #6291 from markhamstra/SPARK-6880.\n\n(cherry picked from commit 0a5aef753e70e93d7e56054f354a52e4d4e18932)\nSigned-off-by: Imran Rashid <irashid@cloudera.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/4139a4ed1f024b1a88e4efe613acf45175f82318"},{"sha":"b1fcefca6b257f5bb1be85fce2b4a97d4271e0aa","author":{"email":"484bb3ee2eae99d0014a3b8fd10f5274ab6daf82@huawei.com","name":"Zhongshuai Pei"},"message":"[SPARK-11974][CORE] Not all the temp dirs had been deleted when the JVM exits\n\ndeleting the temp dir like that\n\n```\n\nscala> import scala.collection.mutable\nimport scala.collection.mutable\n\nscala> val a = mutable.Set(1,2,3,4,7,0,8,98,9)\na: scala.collection.mutable.Set[Int] = Set(0, 9, 1, 2, 3, 7, 4, 8, 98)\n\nscala> a.foreach(x => {a.remove(x) })\n\nscala> a.foreach(println(_))\n98\n```\n\nYou may not modify a collection while traversing or iterating over it.This can not delete all element of the collection\n\nAuthor: Zhongshuai Pei <peizhongshuai@huawei.com>\n\nCloses #9951 from DoingDone9/Bug_RemainDir.\n\n(cherry picked from commit 6b781576a15d8d5c5fbed8bef1c5bda95b3d44ac)\nSigned-off-by: Reynold Xin <rxin@databricks.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/b1fcefca6b257f5bb1be85fce2b4a97d4271e0aa"}]}