{"push_id":688753007,"size":254,"distinct_size":203,"ref":"refs/heads/SPARK-7780-intercept-in-logisticregressionwithLBFGS-should-not-be-regularized","head":"3ac02d72cab72b35b7cc76c50d7088d4b98bfd9d","before":"ed351ffdf862994389b41284f95aa148c6550f41","commits":[{"sha":"3c1a2d049cd4bf35fd48a032f5008b7bab60833e","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"[SPARK-7287] [HOTFIX] Disable o.a.s.deploy.SparkSubmitSuite --packages","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/3c1a2d049cd4bf35fd48a032f5008b7bab60833e"},{"sha":"bfbc0df72944fe0a900ab920d8c4329a11fccca6","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-7845] [BUILD] Bump \"Hadoop 1\" tests to version 1.2.1\n\nhttps://issues.apache.org/jira/browse/SPARK-7845\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #6384 from yhuai/hadoop1Test and squashes the following commits:\n\n82fcea8 [Yin Huai] Use hadoop 1.2.1 (a stable version) for hadoop 1 test.","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/bfbc0df72944fe0a900ab920d8c4329a11fccca6"},{"sha":"ed21476bc0c760616e7e6bb99f6541745fb09595","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-7805] [SQL] Move SQLTestUtils.scala and ParquetTest.scala to src/test\n\nhttps://issues.apache.org/jira/browse/SPARK-7805\n\nBecause `sql/hive`'s tests depend on the test jar of `sql/core`, we do not need to store `SQLTestUtils` and `ParquetTest` in `src/main`. We should only add stuff that will be needed by `sql/console` or Python tests (for Python, we need it in `src/main`, right? davies).\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #6334 from yhuai/SPARK-7805 and squashes the following commits:\n\naf6d0c9 [Yin Huai] mima\nb86746a [Yin Huai] Move SQLTestUtils.scala and ParquetTest.scala to src/test.","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/ed21476bc0c760616e7e6bb99f6541745fb09595"},{"sha":"65c696ecc0a913bbe1c8b8399d811da87e4c4343","author":{"email":"89674bb0dc6f2e03b27640b006f61a9dd0e29d81@hw11853.local","name":"Ram Sriharsha"},"message":"[SPARK-7833] [ML] Add python wrapper for RegressionEvaluator\n\nAuthor: Ram Sriharsha <rsriharsha@hw11853.local>\n\nCloses #6365 from harsha2010/SPARK-7833 and squashes the following commits:\n\n923f288 [Ram Sriharsha] cleanup\n7623b7d [Ram Sriharsha] python style fix\n9743f83 [Ram Sriharsha] [SPARK-7833][ml] Add python wrapper for RegressionEvaluator","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/65c696ecc0a913bbe1c8b8399d811da87e4c4343"},{"sha":"4f4ba8fda8285f5f23143eafcd15be7f4a050122","author":{"email":"b3a1b00f52854a11877d3e6495f2468480cd778b@microsoft.com","name":"Judy Nash"},"message":"[SPARK-7811] Fix typo on slf4j configuration on metrics.properties.tem…\n\nFix minor typo on metrics.properties.template where slf4j is incorrectly spelled as sl4j.\n\nAuthor: Judy Nash <judynash@microsoft.com>\n\nCloses #6362 from judynash/master and squashes the following commits:\n\nc644875 [Judy Nash] SPARK-7811: Fix typo on slf4j configuration on metrics.properties.template","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/4f4ba8fda8285f5f23143eafcd15be7f4a050122"},{"sha":"23bea97d922fbd131d9eb9531906a68459346720","author":{"email":"22f3e725b314afeca4d45e12683c560674029d21@gmail.com","name":"tedyu"},"message":"Close HBaseAdmin at the end of HBaseTest\n\nAuthor: tedyu <yuzhihong@gmail.com>\n\nCloses #6381 from ted-yu/master and squashes the following commits:\n\ne2f0ea1 [tedyu] Close HBaseAdmin at the end of HBaseTest","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/23bea97d922fbd131d9eb9531906a68459346720"},{"sha":"fd31fd49763f7c60b47078c5c2d4b515c123d883","author":{"email":"22f3e725b314afeca4d45e12683c560674029d21@gmail.com","name":"tedyu"},"message":"Add test which shows Kryo buffer size configured in mb is properly supported\n\nThis PR adds test which shows that Kryo buffer size configured in mb is supported properly\n\nAuthor: tedyu <yuzhihong@gmail.com>\n\nCloses #6390 from tedyu/master and squashes the following commits:\n\nc51ea64 [tedyu] Fix KryoSerializer creation\nf12ee04 [tedyu] Correct conf variable name in test\n642de51 [tedyu] Drop change in KryoSerializer so that the new test runs\nd2fdbc4 [tedyu] Give bufferSizeKb initial value\n9a17277 [tedyu] Rewrite bufferSize checking\n4739998 [tedyu] Rewrite bufferSize checking\n830d0d0 [tedyu] Kryo buffer size configured in mb should be properly supported","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/fd31fd49763f7c60b47078c5c2d4b515c123d883"},{"sha":"bfeedc69a29a1dfbfc520545e3fc95389ea1b82d","author":{"email":"e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","name":"Cheng Lian"},"message":"[SPARK-7684] [SQL] Invoking HiveContext.newTemporaryConfiguration() shouldn't create new metastore directory\n\nThe \"Database does not exist\" error reported in SPARK-7684 was caused by `HiveContext.newTemporaryConfiguration()`, which always creates a new temporary metastore directory and returns a metastore configuration pointing that directory. This makes `TestHive.reset()` always replaces old temporary metastore with an empty new one.\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #6359 from liancheng/spark-7684 and squashes the following commits:\n\n95d2eb8 [Cheng Lian] Addresses @marmbrust's comment\n042769d [Cheng Lian] Don't create new temp directory in HiveContext.newTemporaryConfiguration()","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/bfeedc69a29a1dfbfc520545e3fc95389ea1b82d"},{"sha":"8af1bf10b70b9b67f18f618174e84365d69caa48","author":{"email":"e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","name":"Cheng Lian"},"message":"[SPARK-7842] [SQL] Makes task committing/aborting in InsertIntoHadoopFsRelation more robust\n\nWhen committing/aborting a write task issued in `InsertIntoHadoopFsRelation`, if an exception is thrown from `OutputWriter.close()`, the committing/aborting process will be interrupted, and leaves messy stuff behind (e.g., the `_temporary` directory created by `FileOutputCommitter`).\n\nThis PR makes these two process more robust by catching potential exceptions and falling back to normal task committment/abort.\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #6378 from liancheng/spark-7838 and squashes the following commits:\n\nf18253a [Cheng Lian] Makes task committing/aborting in InsertIntoHadoopFsRelation more robust","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/8af1bf10b70b9b67f18f618174e84365d69caa48"},{"sha":"ce0051d6f7e4ed54076676644c78b52b527ba190","author":{"email":"2d9aeb32a8342495c775846b8b174f850c2ddf16@gmail.com","name":"Calvin Jia"},"message":"[SPARK-6391][DOCS] Document Tachyon compatibility.\n\nAdds a section in the RDD persistence section of the programming-guide docs detailing Spark-Tachyon version compatibility as discussed in [[SPARK-6391]](https://issues.apache.org/jira/browse/SPARK-6391).\n\nAuthor: Calvin Jia <jia.calvin@gmail.com>\n\nCloses #6382 from calvinjia/spark-6391 and squashes the following commits:\n\n113e863 [Calvin Jia] Move compatibility info to the offheap storage level section.\n7942dc5 [Calvin Jia] Add a section in the programming-guide docs for Tachyon compatibility.","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/ce0051d6f7e4ed54076676644c78b52b527ba190"},{"sha":"f38e619c41d242143c916373f2a44ec674679f19","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-7832] [Build] Always run SQL tests in master build.\n\nhttps://issues.apache.org/jira/browse/SPARK-7832\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #6385 from yhuai/runSQLTests and squashes the following commits:\n\n3d399bc [Yin Huai] Always run SQL tests in master build.","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/f38e619c41d242143c916373f2a44ec674679f19"},{"sha":"c9adcad81a493595d075715efb2ae395d88a9ec2","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SQL][minor] Removed unused Catalyst logical plan DSL.\n\nThe Catalyst DSL is no longer used as a public facing API. This pull request removes the UDF and writeToFile feature from it since they are not used in unit tests.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #6350 from rxin/unused-logical-dsl and squashes the following commits:\n\n90b3de6 [Reynold Xin] [SQL][minor] Removed unused Catalyst logical plan DSL.","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/c9adcad81a493595d075715efb2ae395d88a9ec2"},{"sha":"43aa819c041f6e8301ad1b8f82eb68e14254f636","author":{"email":"b98bbd1d47292cd44bd0779957153c7a22567181@sc.com","name":"Konstantin Shaposhnikov"},"message":"[SPARK-7042] [BUILD] use the standard akka artifacts with hadoop-2.x\n\nBoth akka 2.3.x and hadoop-2.x use protobuf 2.5 so only hadoop-1 build needs\ncustom 2.3.4-spark akka version that shades protobuf-2.5\n\nThis partially fixes SPARK-7042 (for hadoop-2.x builds)\n\nAuthor: Konstantin Shaposhnikov <Konstantin.Shaposhnikov@sc.com>\n\nCloses #6341 from kostya-sh/SPARK-7042 and squashes the following commits:\n\n7eb8c60 [Konstantin Shaposhnikov] [SPARK-7042][BUILD] use the standard akka artifacts with hadoop-2.x","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/43aa819c041f6e8301ad1b8f82eb68e14254f636"},{"sha":"bf49c22130af9a729dcc510743e4c1ea4c5d2439","author":{"email":"f98b0bead8ca0151df080a83631a6c2556256922@huawei.com","name":"scwf"},"message":"[CORE] [TEST] Fix SimpleDateParamTest\n\n```\nsbt.ForkMain$ForkError: 1424424077190 was not equal to 1424474477190\n\tat org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:160)\n\tat org.scalatest.Matchers$ShouldMethodHelper$.shouldMatcher(Matchers.scala:6231)\n\tat org.scalatest.Matchers$AnyShouldWrapper.should(Matchers.scala:6265)\n\tat org.apache.spark.status.api.v1.SimpleDateParamTest$$anonfun$1.apply$mcV$sp(SimpleDateParamTest.scala:25)\n\tat org.apache.spark.status.api.v1.SimpleDateParamTest$$anonfun$1.apply(SimpleDateParamTest.scala:23)\n\tat org.apache.spark.status.api.v1.SimpleDateParamTest$$anonfun$1.apply(SimpleDateParamTest.scala:23)\n\tat org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)\n\tat org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)\n\tat org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)\n\tat org.scalatest.Transformer.apply(Transformer.scala:22)\n\tat org.scalatest.Transformer.apply(Transformer.scala:20)\n\tat org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)\n\tat org.scalatest.Suite$class.withFixture(Suite.scala:\n```\n\nSet timezone to fix SimpleDateParamTest\n\nAuthor: scwf <wangfei1@huawei.com>\nAuthor: Fei Wang <wangfei1@huawei.com>\n\nCloses #6377 from scwf/fix-SimpleDateParamTest and squashes the following commits:\n\nb8df1e5 [Fei Wang] Update SimpleDateParamSuite.scala\n8bb74f0 [scwf] fix SimpleDateParamSuite","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/bf49c22130af9a729dcc510743e4c1ea4c5d2439"},{"sha":"8948ad3fb5d5d095d3942855960d735f27d97dd5","author":{"email":"c40433a0b9f1df8d6cfdcc84006d6c506907d641@huawei.com","name":"linweizhong"},"message":"[SPARK-7339] [PYSPARK] PySpark shuffle spill memory sometimes are not correct\n\nIn PySpark we get memory used before and after spill, then use the difference of these two value as memorySpilled, but if the before value is small than after value, then we will get a negative value, but this scenario 0 value may be more reasonable.\n\nBelow is the result in HistoryServer we have tested:\nIndex\tID\tAttempt\tStatus\tLocality Level\tExecutor ID / Host\tLaunch Time\tDuration\tGC Time\tInput Size / Records\tWrite Time\tShuffle Write Size / Records\tShuffle Spill (Memory)\tShuffle Spill (Disk)\tErrors\n0\t0\t0\tSUCCESS\tNODE_LOCAL\t3 / vm119\t2015/05/04 17:31:06\t21 s\t0.1 s\t128.1 MB (hadoop) / 3237\t70 ms\t10.1 MB / 2529\t0.0 B\t5.7 MB\n2\t2\t0\tSUCCESS\tNODE_LOCAL\t1 / vm118\t2015/05/04 17:31:06\t22 s\t89 ms\t128.1 MB (hadoop) / 3205\t0.1 s\t10.1 MB / 2529\t-1048576.0 B\t5.9 MB\n1\t1\t0\tSUCCESS\tNODE_LOCAL\t2 / vm117\t2015/05/04 17:31:06\t22 s\t0.1 s\t128.1 MB (hadoop) / 3271\t68 ms\t10.1 MB / 2529\t-1048576.0 B\t5.6 MB\n4\t4\t0\tSUCCESS\tNODE_LOCAL\t2 / vm117\t2015/05/04 17:31:06\t22 s\t0.1 s\t128.1 MB (hadoop) / 3192\t51 ms\t10.1 MB / 2529\t-1048576.0 B\t5.9 MB\n3\t3\t0\tSUCCESS\tNODE_LOCAL\t3 / vm119\t2015/05/04 17:31:06\t22 s\t0.1 s\t128.1 MB (hadoop) / 3262\t51 ms\t10.1 MB / 2529\t1024.0 KB\t5.8 MB\n5\t5\t0\tSUCCESS\tNODE_LOCAL\t1 / vm118\t2015/05/04 17:31:06\t22 s\t89 ms\t128.1 MB (hadoop) / 3256\t93 ms\t10.1 MB / 2529\t-1048576.0 B\t5.7 MB\n\n/cc davies\n\nAuthor: linweizhong <linweizhong@huawei.com>\n\nCloses #5887 from Sephiroth-Lin/spark-7339 and squashes the following commits:\n\n9186c81 [linweizhong] Use max function to get a nonnegative value\nd41672b [linweizhong] Update MemoryBytesSpilled when memorySpilled > 0","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/8948ad3fb5d5d095d3942855960d735f27d97dd5"},{"sha":"8dbe777703e0aaf47cbdfe98f66d22f723352fb5","author":{"email":"6e126cc70b7baccb2cfe4f74f6527dabde0005f5@gmail.com","name":"meawoppl"},"message":"[SPARK-7806][EC2] Fixes that allow the spark_ec2.py tool to run with Python3\n\nI have used this script to launch, destroy, start, and stop clusters successfully.\n\nAuthor: meawoppl <meawoppl@gmail.com>\n\nCloses #6336 from meawoppl/py3ec2spark and squashes the following commits:\n\n2e87046 [meawoppl] Py3 compat fixes.","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/8dbe777703e0aaf47cbdfe98f66d22f723352fb5"},{"sha":"e5a63a0e39e35ba5f6b1732e09128c30fff17a2c","author":{"email":"fcf9e35ad50aab44387f6ff709c4c4bf73ee4abe@gmail.com","name":"Mike Dusenberry"},"message":"[DOCS] [MLLIB] Fixing misformatted links in v1.4 MLlib Naive Bayes documentation by removing space and newline characters.\n\nA couple of links in the MLlib Naive Bayes documentation for v1.4 were broken due to the addition of either space or newline characters between the link title and link URL in the markdown doc.  (Interestingly enough, they are rendered correctly in the GitHub viewer, but not when compiled to HTML by Jekyll.)\n\nAuthor: Mike Dusenberry <dusenberrymw@gmail.com>\n\nCloses #6412 from dusenberrymw/Fix_Broken_Links_In_MLlib_Naive_Bayes_Docs and squashes the following commits:\n\n91a4028 [Mike Dusenberry] Fixing misformatted links by removing space and newline characters.","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/e5a63a0e39e35ba5f6b1732e09128c30fff17a2c"},{"sha":"63099122deb93553f5a03bb3ff74e7c2f1ee2164","author":{"email":"7632d88ebdd73175d662531eefd9a84b8ade5e24@intel.com","name":"Zhang, Liye"},"message":"[SPARK-7854] [TEST] refine Kryo test suite\n\nthis modification is according to JoshRosen 's comments, for details, please refer to [#5934](https://github.com/apache/spark/pull/5934/files#r30949751).\n\nAuthor: Zhang, Liye <liye.zhang@intel.com>\n\nCloses #6395 from liyezhang556520/kryoTest and squashes the following commits:\n\nda214c8 [Zhang, Liye] refine Kryo test suite accroding to Josh's comments","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/63099122deb93553f5a03bb3ff74e7c2f1ee2164"},{"sha":"b7d8085942c564d6c5b81a14d31789e1b215e62b","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"Revert \"[SPARK-7042] [BUILD] use the standard akka artifacts with hadoop-2.x\"\n\nThis reverts commit 43aa819c041f6e8301ad1b8f82eb68e14254f636.","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/b7d8085942c564d6c5b81a14d31789e1b215e62b"},{"sha":"61664732b25b35f94be35a42cde651cbfd0e02b7","author":{"email":"8757c4a255384de9fc2175f17a84f948e7081d30@gmail.com","name":"MechCoder"},"message":"[SPARK-7844] [MLLIB] Fix broken tests in KernelDensity\n\nThe densities in KernelDensity are scaled down by\n(number of parallel processes X number of points). It should be just no.of samples. This results in broken tests in KernelDensitySuite which haven't been tested properly.\n\nAuthor: MechCoder <manojkumarsivaraj334@gmail.com>\n\nCloses #6383 from MechCoder/spark-7844 and squashes the following commits:\n\nab81302 [MechCoder] Math->math\n9b8ed50 [MechCoder] Make one pass to update count\na92fe50 [MechCoder] [SPARK-7844] Fix broken tests in KernelDensity","distinct":false,"url":"https://api.github.com/repos/holdenk/spark/commits/61664732b25b35f94be35a42cde651cbfd0e02b7"}]}