{"push_id":768801400,"size":14,"distinct_size":14,"ref":"refs/heads/master","head":"d7b4c095271c36fcc7f9ded267ecf5ec66fac803","before":"630a994e6a9785d1704f8e7fb604f32f5dea24f8","commits":[{"sha":"46fcb9e0dbb2b28110f68a3d9f6c0c47bfd197b1","author":{"email":"38bfc72e6a35a427ed65527be3b49f08e9586edf@gmail.com","name":"Keiji Yoshida"},"message":"Update programming-guide.md\n\nUpdate `lineLengths.persist();` to `lineLengths.persist(StorageLevel.MEMORY_ONLY());` because `JavaRDD#persist` needs a parameter of `StorageLevel`.\n\nAuthor: Keiji Yoshida <yoshida.keiji.84@gmail.com>\n\nCloses #8372 from yosssi/patch-1.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/46fcb9e0dbb2b28110f68a3d9f6c0c47bfd197b1"},{"sha":"90cb9f05655a25b95b8f9fe81da14e5b9c8bcf44","author":{"email":"348ff379544b45aa3f16e53a543620141423a0f6@gmail.com","name":"Yijie Shen"},"message":"[SPARK-9401] [SQL] Fully implement code generation for ConcatWs\n\nThis PR adds full codegen support for ConcatWs, is a substitute of #7782\n\nJIRA: https://issues.apache.org/jira/browse/SPARK-9401\n\ncc davies\n\nAuthor: Yijie Shen <henry.yijieshen@gmail.com>\n\nCloses #8353 from yjshen/concatws.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/90cb9f05655a25b95b8f9fe81da14e5b9c8bcf44"},{"sha":"623c675fde7a3a39957a62c7af26a54f4b01f8ce","author":{"email":"38bfc72e6a35a427ed65527be3b49f08e9586edf@gmail.com","name":"Keiji Yoshida"},"message":"Update streaming-programming-guide.md\n\nUpdate `See the Scala example` to `See the Java example`.\n\nAuthor: Keiji Yoshida <yoshida.keiji.84@gmail.com>\n\nCloses #8376 from yosssi/patch-1.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/623c675fde7a3a39957a62c7af26a54f4b01f8ce"},{"sha":"c6df5f66d9a8b9760f2cd46fcd930f977650c9c5","author":{"email":"5d14e010a5b345b784fd604efcc6cebb79f97468@gmail.com","name":"zsxwing"},"message":"[SPARK-10148] [STREAMING] Display active and inactive receiver numbers in Streaming page\n\nAdded the active and inactive receiver numbers in the summary section of Streaming page.\n\n<img width=\"1074\" alt=\"screen shot 2015-08-21 at 2 08 54 pm\" src=\"https://cloud.githubusercontent.com/assets/1000778/9402437/ff2806a2-480f-11e5-8f8e-efdf8e5d514d.png\">\n\nAuthor: zsxwing <zsxwing@gmail.com>\n\nCloses #8351 from zsxwing/receiver-number.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/c6df5f66d9a8b9760f2cd46fcd930f977650c9c5"},{"sha":"b963c19a803c5a26c9b65655d40ca6621acf8bd4","author":{"email":"461476587780aa9fa5611ea6dc3912c146a91760@databricks.com","name":"Joseph K. Bradley"},"message":"[SPARK-10164] [MLLIB] Fixed GMM distributed decomposition bug\n\nGaussianMixture now distributes matrix decompositions for certain problem sizes. Distributed computation actually fails, but this was not tested in unit tests.\n\nThis PR adds a unit test which checks this.  It failed previously but works with this fix.\n\nCC: mengxr\n\nAuthor: Joseph K. Bradley <joseph@databricks.com>\n\nCloses #8370 from jkbradley/gmm-fix.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/b963c19a803c5a26c9b65655d40ca6621acf8bd4"},{"sha":"053d94fcf32268369b5a40837271f15d6af41aa4","author":{"email":"4330a85689689e23b81e88bdf5853b25c33529eb@gmail.com","name":"Tathagata Das"},"message":"[SPARK-10142] [STREAMING] Made python checkpoint recovery handle non-local checkpoint paths and existing SparkContexts\n\nThe current code only checks checkpoint files in local filesystem, and always tries to create a new Python SparkContext (even if one already exists). The solution is to do the following:\n1. Use the same code path as Java to check whether a valid checkpoint exists\n2. Create a new Python SparkContext only if there no active one.\n\nThere is not test for the path as its hard to test with distributed filesystem paths in a local unit test. I am going to test it with a distributed file system manually to verify that this patch works.\n\nAuthor: Tathagata Das <tathagata.das1565@gmail.com>\n\nCloses #8366 from tdas/SPARK-10142 and squashes the following commits:\n\n3afa666 [Tathagata Das] Added tests\n2dd4ae5 [Tathagata Das] Added the check to not create a context if one already exists\n9bf151b [Tathagata Das] Made python checkpoint recovery use java to find the checkpoint files","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/053d94fcf32268369b5a40837271f15d6af41aa4"},{"sha":"4e0395ddb764d092b5b38447af49e196e590e0f0","author":{"email":"5d14e010a5b345b784fd604efcc6cebb79f97468@gmail.com","name":"zsxwing"},"message":"[SPARK-10168] [STREAMING] Fix the issue that maven publishes wrong artifact jars\n\nThis PR removed the `outputFile` configuration from pom.xml and updated `tests.py` to search jars for both sbt build and maven build.\n\nI ran ` mvn -Pkinesis-asl -DskipTests clean install` locally, and verified the jars in my local repository were correct. I also checked Python tests for maven build, and it passed all tests.\n\nAuthor: zsxwing <zsxwing@gmail.com>\n\nCloses #8373 from zsxwing/SPARK-10168 and squashes the following commits:\n\ne0b5818 [zsxwing] Fix the sbt build\nc697627 [zsxwing] Add the jar pathes to the exception message\nbe1d8a5 [zsxwing] Fix the issue that maven publishes wrong artifact jars","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/4e0395ddb764d092b5b38447af49e196e590e0f0"},{"sha":"7478c8b66d6a2b1179f20c38b49e27e37b0caec3","author":{"email":"4330a85689689e23b81e88bdf5853b25c33529eb@gmail.com","name":"Tathagata Das"},"message":"[SPARK-9791] [PACKAGE] Change private class to private class to prevent unnecessary classes from showing up in the docs\n\nIn addition, some random cleanup of import ordering\n\nAuthor: Tathagata Das <tathagata.das1565@gmail.com>\n\nCloses #8387 from tdas/SPARK-9791 and squashes the following commits:\n\n67f3ee9 [Tathagata Das] Change private class to private[package] class to prevent them from showing up in the docs","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/7478c8b66d6a2b1179f20c38b49e27e37b0caec3"},{"sha":"9ce0c7ad333f4a3c01207e5e9ed42bcafb99d894","author":{"email":"affa8bc277be808c7f93de9249c2a75ded11876a@gmail.com","name":"Burak Yavuz"},"message":"[SPARK-7710] [SPARK-7998] [DOCS] Docs for DataFrameStatFunctions\n\nThis PR contains examples on how to use some of the Stat Functions available for DataFrames under `df.stat`.\n\nrxin\n\nAuthor: Burak Yavuz <brkyvz@gmail.com>\n\nCloses #8378 from brkyvz/update-sql-docs.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/9ce0c7ad333f4a3c01207e5e9ed42bcafb99d894"},{"sha":"662bb9667669cb07cf6d2ccee0d8e76bb561cd89","author":{"email":"02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@databricks.com","name":"Andrew Or"},"message":"[SPARK-10144] [UI] Actually show peak execution memory by default\n\nThe peak execution memory metric was introduced in SPARK-8735. That was before Tungsten was enabled by default, so it assumed that `spark.sql.unsafe.enabled` must be explicitly set to true. The result is that the memory is not displayed by default.\n\nAuthor: Andrew Or <andrew@databricks.com>\n\nCloses #8345 from andrewor14/show-memory-default.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/662bb9667669cb07cf6d2ccee0d8e76bb561cd89"},{"sha":"a2f4cdceba32aaa0df59df335ca0ce1ac73fc6c2","author":{"email":"e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","name":"Cheng Lian"},"message":"[SPARK-8580] [SQL] Refactors ParquetHiveCompatibilitySuite and adds more test cases\n\nThis PR refactors `ParquetHiveCompatibilitySuite` so that it's easier to add new test cases.\n\nHit two bugs, SPARK-10177 and HIVE-11625, while working on this, added test cases for them and marked as ignored for now. SPARK-10177 will be addressed in a separate PR.\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #8392 from liancheng/spark-8580/parquet-hive-compat-tests.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/a2f4cdceba32aaa0df59df335ca0ce1ac73fc6c2"},{"sha":"cb2d2e15844d7ae34b5dd7028b55e11586ed93fa","author":{"email":"291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","name":"Sean Owen"},"message":"[SPARK-9758] [TEST] [SQL] Compilation issue for hive test / wrong package?\n\nMove `test.org.apache.spark.sql.hive` package tests to apparent intended `org.apache.spark.sql.hive` as they don't intend to test behavior from outside org.apache.spark.*\n\nAlternate take, per discussion at https://github.com/apache/spark/pull/8051\nI think this is what vanzin and I had in mind but also CC rxin to cross-check, as this does indeed depend on whether these tests were accidentally in this package or not. Testing from a `test.org.apache.spark` package is legitimate but didn't seem to be the intent here.\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #8307 from srowen/SPARK-9758.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/cb2d2e15844d7ae34b5dd7028b55e11586ed93fa"},{"sha":"13db11cb08eb90eb0ea3402c9fe0270aa282f247","author":{"email":"461476587780aa9fa5611ea6dc3912c146a91760@databricks.com","name":"Joseph K. Bradley"},"message":"[SPARK-10061] [DOC] ML ensemble docs\n\nUser guide for spark.ml GBTs and Random Forests.\nThe examples are copied from the decision tree guide and modified to run.\n\nI caught some issues I had somehow missed in the tree guide as well.\n\nI have run all examples, including Java ones.  (Of course, I thought I had previously as well...)\n\nCC: mengxr manishamde yanboliang\n\nAuthor: Joseph K. Bradley <joseph@databricks.com>\n\nCloses #8369 from jkbradley/ml-ensemble-docs.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/13db11cb08eb90eb0ea3402c9fe0270aa282f247"},{"sha":"d7b4c095271c36fcc7f9ded267ecf5ec66fac803","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-10190] Fix NPE in CatalystTypeConverters Decimal toScala converter\n\nThis adds a missing null check to the Decimal `toScala` converter in `CatalystTypeConverters`, fixing an NPE.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #8401 from JoshRosen/SPARK-10190.","distinct":true,"url":"https://api.github.com/repos/Agent007/spark/commits/d7b4c095271c36fcc7f9ded267ecf5ec66fac803"}]}