{"push_id":630725570,"size":12,"distinct_size":12,"ref":"refs/heads/branch-1.2","head":"daec1c6353e5e4daac2f082f714e45a95939a538","before":"758ebf77d7daded7c5f6f41ee269205bc246d487","commits":[{"sha":"a73055f7f9104cb5a9ed43a6cd4a82d463702b60","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-6667] [PySpark] remove setReuseAddress\n\nThe reused address on server side had caused the server can not acknowledge the connected connections, remove it.\n\nThis PR will retry once after timeout, it also add a timeout at client side.\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #5324 from davies/collect_hang and squashes the following commits:\n\ne5a51a2 [Davies Liu] remove setReuseAddress\n7977c2f [Davies Liu] do retry on client side\nb838f35 [Davies Liu] retry after timeout\n\n(cherry picked from commit 0cce5451adfc6bf4661bcf67aca3db26376455fe)\nSigned-off-by: Josh Rosen <joshrosen@databricks.com>","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/a73055f7f9104cb5a9ed43a6cd4a82d463702b60"},{"sha":"8fa09a480848faf4eda263cc1e79e0dd56a52605","author":{"email":"6cd6c3674446c089a7a8ccdb2f87321bcfb4b435@gmail.com","name":"Hung Lin"},"message":"SPARK-6414: Spark driver failed with NPE on job cancelation\n\nUse Option for ActiveJob.properties to avoid NPE bug\n\nAuthor: Hung Lin <hung.lin@gmail.com>\n\nCloses #5124 from hunglin/SPARK-6414 and squashes the following commits:\n\n2290b6b [Hung Lin] [SPARK-6414][core] Fix NPE in SparkContext.cancelJobGroup()\n\n(cherry picked from commit e3202aa2e9bd140effbcf2a7a02b90cb077e760b)\nSigned-off-by: Josh Rosen <joshrosen@databricks.com>\n\nConflicts:\n\tcore/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala\n\nConflicts:\n\tcore/src/test/scala/org/apache/spark/SparkContextSuite.scala","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/8fa09a480848faf4eda263cc1e79e0dd56a52605"},{"sha":"d82e73239118fe99535eaa68be9bf37e837bebe4","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":" [SPARK-6578] [core] Fix thread-safety issue in outbound path of network library.\n\nWhile the inbound path of a netty pipeline is thread-safe, the outbound\npath is not. That means that multiple threads can compete to write messages\nto the next stage of the pipeline.\n\nThe network library sometimes breaks a single RPC message into multiple\nbuffers internally to avoid copying data (see MessageEncoder). This can\nresult in the following scenario (where \"FxBy\" means \"frame x, buffer y\"):\n\n               T1         F1B1            F1B2\n                            \\               \\\n                             \\               \\\n               socket        F1B1   F2B1    F1B2  F2B2\n                                     /             /\n                                    /             /\n               T2                  F2B1         F2B2\n\nAnd the frames now cannot be rebuilt on the receiving side because the\ndifferent messages have been mixed up on the wire.\n\nThe fix wraps these multi-buffer messages into a `FileRegion` object\nso that these messages are written \"atomically\" to the next pipeline handler.\n\nAuthor: Reynold Xin <rxin@databricks.com>\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #5336 from vanzin/SPARK-6578-1.2 and squashes the following commits:\n\n4d3395e [Reynold Xin] [SPARK-6578] Small rewrite to make the logic more clear in MessageWithHeader.transferTo.\n526f230 [Marcelo Vanzin] [SPARK-6578] [core] Fix thread-safety issue in outbound path of network library.","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/d82e73239118fe99535eaa68be9bf37e837bebe4"},{"sha":"2991dd04215a4ba5c3367b71ff19de47efae47ca","author":{"email":"ff731187407040232c9d93c77d0fa88c791883e8@qq.com","name":"seayi"},"message":"[SPARK-5195][sql]Update HiveMetastoreCatalog.scala(override the MetastoreRelation's sameresult method only compare databasename and table name)\n\noverride  the MetastoreRelation's  sameresult method only compare databasename and table name\n\nbecause in previous :\ncache table t1;\nselect count(*) from t1;\nit will read data from memory  but the sql below will not,instead it read from hdfs:\nselect count(*) from t1 t;\n\nbecause cache data is keyed by logical plan and compare with sameResult ,so  when table with alias  the same table 's logicalplan is not the same logical plan with out alias  so modify  the sameresult method only compare databasename and table name\n\nAuthor: seayi <405078363@qq.com>\nAuthor: Michael Armbrust <michael@databricks.com>\n\nCloses #3898 from seayi/branch-1.2 and squashes the following commits:\n\n8f0c7d2 [seayi] Update CachedTableSuite.scala\na277120 [seayi] Update HiveMetastoreCatalog.scala\n8d910aa [seayi] Update HiveMetastoreCatalog.scala","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/2991dd04215a4ba5c3367b71ff19de47efae47ca"},{"sha":"f4a9c41b1386b78d7fbf3fa37b0b424da528a17d","author":{"email":"1bbfbf3ad5c4de88db64d0be3ca037b409d25231@huawei.com","name":"KaiXinXiaoLei"},"message":"[CORE] The descriptionof jobHistory config should be spark.history.fs.logDirectory\n\nThe config option  is spark.history.fs.logDirectory, not spark.fs.history.logDirectory. So the descriptionof  should be changed. Thanks.\n\nAuthor: KaiXinXiaoLei <huleilei1@huawei.com>\n\nCloses #5332 from KaiXinXiaoLei/historyConfig and squashes the following commits:\n\n5ffbfb5 [KaiXinXiaoLei] the describe of jobHistory config is error\n\n(cherry picked from commit 8a0aa81ca37d337423db60edb09cf264cc2c6498)\nSigned-off-by: Andrew Or <andrew@databricks.com>","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/f4a9c41b1386b78d7fbf3fa37b0b424da528a17d"},{"sha":"eac952503a1c4b188c1839aca29fada92f605147","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"[HOTFIX] Updating CHANGES.txt for Spark 1.2.2","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/eac952503a1c4b188c1839aca29fada92f605147"},{"sha":"86d1715cd44186a43b8676d48e995d82b215581e","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"[HOTFIX] Bumping versions for Spark 1.2.2","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/86d1715cd44186a43b8676d48e995d82b215581e"},{"sha":"7531b50e406ee2e3301b009ceea7c684272b2e27","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"Preparing Spark release v1.2.2-rc1","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/7531b50e406ee2e3301b009ceea7c684272b2e27"},{"sha":"7b7db5942481e3878901c22010dd1cb77ccdda0e","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"Preparing development version 1.2.3-SNAPSHOT","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/7b7db5942481e3878901c22010dd1cb77ccdda0e"},{"sha":"f7fe87f4b71f469f3c67a00883697ff09cc8a96a","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-6209] Clean up connections in ExecutorClassLoader after failing to load classes (branch-1.2)\n\nExecutorClassLoader does not ensure proper cleanup of network connections that it opens. If it fails to load a class, it may leak partially-consumed InputStreams that are connected to the REPL's HTTP class server, causing that server to exhaust its thread pool, which can cause the entire job to hang.  See [SPARK-6209](https://issues.apache.org/jira/browse/SPARK-6209) for more details, including a bug reproduction.\n\nThis patch fixes this issue by ensuring proper cleanup of these resources.  It also adds logging for unexpected error cases.\n\n(See #4944 for the corresponding PR for 1.3/1.4).\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #5174 from JoshRosen/executorclassloaderleak-branch-1.2 and squashes the following commits:\n\n16e38fe [Josh Rosen] [SPARK-6209] Clean up connections in ExecutorClassLoader after failing to load classes (master branch PR)","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/f7fe87f4b71f469f3c67a00883697ff09cc8a96a"},{"sha":"7a1583917e4a73a464d3db57406fe708a9283c7c","author":{"email":"284a00ea9901227c42561196ff8576aed3ac5631@gmail.com","name":"Kay Ousterhout"},"message":"[SPARK-6753] Clone SparkConf in ShuffleSuite tests\n\nPrior to this change, the unit test for SPARK-3426 did not clone the\noriginal SparkConf, which meant that that test did not use the options\nset by suites that subclass ShuffleSuite.scala. This commit fixes that\nproblem.\n\nJoshRosen would be great if you could take a look at this, since you wrote this\ntest originally.\n\nAuthor: Kay Ousterhout <kayousterhout@gmail.com>\n\nCloses #5401 from kayousterhout/SPARK-6753 and squashes the following commits:\n\n368c540 [Kay Ousterhout] [SPARK-6753] Clone SparkConf in ShuffleSuite tests\n\n(cherry picked from commit 9d44ddce1d1e19011026605549c37d0db6d6afa1)\nSigned-off-by: Josh Rosen <joshrosen@databricks.com>","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/7a1583917e4a73a464d3db57406fe708a9283c7c"},{"sha":"daec1c6353e5e4daac2f082f714e45a95939a538","author":{"email":"ff0f0a8b656f0b44c26933acd2e367b6c1211290@ucw.cz","name":"Milan Straka"},"message":"[SPARK-5969][PySpark] Fix descending pyspark.rdd.sortByKey.\n\nThe samples should always be sorted in ascending order, because bisect.bisect_left is used on it. The reverse order of the result is already achieved in rangePartitioner by reversing the found index.\n\nThe current implementation also work, but always uses only two partitions -- the first one and the last one (because the bisect_left return returns either \"beginning\" or \"end\" for a descending sequence).\n\nAuthor: Milan Straka <fox@ucw.cz>\n\nThis patch had conflicts when merged, resolved by\nCommitter: Josh Rosen <joshrosen@databricks.com>\n\nCloses #4761 from foxik/fix-descending-sort and squashes the following commits:\n\n95896b5 [Milan Straka] Add regression test for SPARK-5969.\n5757490 [Milan Straka] Fix descending pyspark.rdd.sortByKey.","distinct":true,"url":"https://api.github.com/repos/chuckchen/spark/commits/daec1c6353e5e4daac2f082f714e45a95939a538"}]}