{"push_id":766607153,"size":92,"distinct_size":1,"ref":"refs/heads/master","head":"6e899884eefa994e130652d99b89a0b1628b3ed0","before":"f10660fe7b809be2059da4f9781a5743f117a35a","commits":[{"sha":"b265e282b62954548740a5767e97ab1678c65194","author":{"email":"348ff379544b45aa3f16e53a543620141423a0f6@gmail.com","name":"Yijie Shen"},"message":"[SPARK-9526] [SQL] Utilize randomized tests to reveal potential bugs in sql expressions\n\nJIRA: https://issues.apache.org/jira/browse/SPARK-9526\n\nThis PR is a follow up of #7830, aiming at utilizing randomized tests to reveal more potential bugs in sql expression.\n\nAuthor: Yijie Shen <henry.yijieshen@gmail.com>\n\nCloses #7855 from yjshen/property_check.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/b265e282b62954548740a5767e97ab1678c65194"},{"sha":"772e7c18fb1a79c0f080408cb43307fe89a4fa04","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-9592] [SQL] Fix Last function implemented based on AggregateExpression1.\n\nhttps://issues.apache.org/jira/browse/SPARK-9592\n\n#8113 has the fundamental fix. But, if we want to minimize the number of changed lines, we can go with this one. Then, in 1.6, we merge #8113.\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #8172 from yhuai/lastFix and squashes the following commits:\n\nb28c42a [Yin Huai] Regression test.\naf87086 [Yin Huai] Fix last.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/772e7c18fb1a79c0f080408cb43307fe89a4fa04"},{"sha":"fdaf17f63f751f02623414fbc7d0a2f545364050","author":{"email":"9835e449f9a666ddd8ba8aeb460af39a5130691b@databricks.com","name":"Feynman Liang"},"message":"[SPARK-10068] [MLLIB] Adds links to MLlib types, algos, utilities listing\n\nmengxr jkbradley\n\nAuthor: Feynman Liang <fliang@databricks.com>\n\nCloses #8255 from feynmanliang/SPARK-10068.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/fdaf17f63f751f02623414fbc7d0a2f545364050"},{"sha":"088b11ec5949e135cb3db2a1ce136837e046c288","author":{"email":"f343088f0cf5f2dc6d28ebac254bf6ea7a2528e2@sabhyankar-MBP.Samavihome","name":"Sameer Abhyankar"},"message":"[SPARK-8920] [MLLIB] Add @since tags to mllib.linalg\n\nAuthor: Sameer Abhyankar <sabhyankar@sabhyankar-MBP.Samavihome>\nAuthor: Sameer Abhyankar <sabhyankar@sabhyankar-MBP.local>\n\nCloses #7729 from sabhyankar/branch_8920.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/088b11ec5949e135cb3db2a1ce136837e046c288"},{"sha":"52ae952574f5d641a398dd185e09e5a79318c8a9","author":{"email":"e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","name":"Cheng Lian"},"message":"[SPARK-9974] [BUILD] [SQL] Makes sure com.twitter:parquet-hadoop-bundle:1.6.0 is in SBT assembly jar\n\nPR #7967 enables Spark SQL to persist Parquet tables in Hive compatible format when possible. One of the consequence is that, we have to set input/output classes to `MapredParquetInputFormat`/`MapredParquetOutputFormat`, which rely on com.twitter:parquet-hadoop:1.6.0 bundled with Hive 1.2.1.\n\nWhen loading such a table in Spark SQL, `o.a.h.h.ql.metadata.Table` first loads these input/output format classes, and thus classes in com.twitter:parquet-hadoop:1.6.0.  However, the scope of this dependency is defined as \"runtime\", and is not packaged into Spark assembly jar.  This results in a `ClassNotFoundException`.\n\nThis issue can be worked around by asking users to add parquet-hadoop 1.6.0 via the `--driver-class-path` option.  However, considering Maven build is immune to this problem, I feel it can be confusing and inconvenient for users.\n\nSo this PR fixes this issue by changing scope of parquet-hadoop 1.6.0 to \"compile\".\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #8198 from liancheng/spark-9974/bundle-parquet-1.6.0.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/52ae952574f5d641a398dd185e09e5a79318c8a9"},{"sha":"0076e8212334c613599dcbc2ac23f49e9e50cc44","author":{"email":"452ae006f36be6c70342f33f64bb6f6eb552689c@gmail.com","name":"Yanbo Liang"},"message":"[SPARK-9768] [PYSPARK] [ML] Add Python API and user guide for ml.feature.ElementwiseProduct\n\nAdd Python API, user guide and example for ml.feature.ElementwiseProduct.\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #8061 from yanboliang/SPARK-9768.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/0076e8212334c613599dcbc2ac23f49e9e50cc44"},{"sha":"18523c130548f0438dff8d1f25531fd2ed36e517","author":{"email":"631741d0dfe088451ff3c2939a3886b1e6dfed33@gmail.com","name":"Prayag Chandran"},"message":"SPARK-8916 [Documentation, MLlib] Add @since tags to mllib.regression\n\nAdded since tags to mllib.regression\n\nAuthor: Prayag Chandran <prayagchandran@gmail.com>\n\nCloses #7518 from prayagchandran/sinceTags and squashes the following commits:\n\nfa4dda2 [Prayag Chandran] Re-formatting\n6c6d584 [Prayag Chandran] Corrected a few tags. Removed few unnecessary tags\n1a0365f [Prayag Chandran] Reformating and adding a few more tags\n89fdb66 [Prayag Chandran] SPARK-8916 [Documentation, MLlib] Add @since tags to mllib.regression","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/18523c130548f0438dff8d1f25531fd2ed36e517"},{"sha":"0b6b01761370629ce387c143a25d41f3a334ff28","author":{"email":"9835e449f9a666ddd8ba8aeb460af39a5130691b@databricks.com","name":"Feynman Liang"},"message":"[SPARK-9898] [MLLIB] Prefix Span user guide\n\nAdds user guide for `PrefixSpan`, including Scala and Java example code.\n\nmengxr zhangjiajin\n\nAuthor: Feynman Liang <fliang@databricks.com>\n\nCloses #8253 from feynmanliang/SPARK-9898.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/0b6b01761370629ce387c143a25d41f3a334ff28"},{"sha":"f9d1a92aa1bac4494022d78559b871149579e6e8","author":{"email":"5673041ad99da73cfefa170a7081c0daed5d97b6@cloudera.com","name":"Sandy Ryza"},"message":"[SPARK-7707] User guide and example code for KernelDensity\n\nAuthor: Sandy Ryza <sandy@cloudera.com>\n\nCloses #8230 from sryza/sandy-spark-7707.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/f9d1a92aa1bac4494022d78559b871149579e6e8"},{"sha":"c90c605dc6a876aef3cc204ac15cd65bab9743ad","author":{"email":"dd19fa80624ab884ac33cf50a82d9bdcaeb1d7b3@cloudera.com","name":"jose.cambronero"},"message":"[SPARK-9902] [MLLIB] Add Java and Python examples to user guide for 1-sample KS test\n\nadded doc examples for python.\n\nAuthor: jose.cambronero <jose.cambronero@cloudera.com>\n\nCloses #8154 from josepablocam/spark_9902.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/c90c605dc6a876aef3cc204ac15cd65bab9743ad"},{"sha":"ee093c8b927e8d488aeb76115c7fb0de96af7720","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-10059] [YARN] Explicitly add JSP dependencies for tests.\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #8251 from vanzin/SPARK-10059.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/ee093c8b927e8d488aeb76115c7fb0de96af7720"},{"sha":"e290029a356222bddf4da1be0525a221a5a1630b","author":{"email":"9616cf314f60cbadfd3d5d51110130ad7923defb@databricks.com","name":"Xiangrui Meng"},"message":"[SPARK-7808] [ML] add package doc for ml.feature\n\nThis PR adds a short description of `ml.feature` package with code example. The Java package doc will come in a separate PR. jkbradley\n\nAuthor: Xiangrui Meng <meng@databricks.com>\n\nCloses #8260 from mengxr/SPARK-7808.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/e290029a356222bddf4da1be0525a221a5a1630b"},{"sha":"a0910315dae88b033e38a1de07f39ca21f6552ad","author":{"email":"bc4a65f227ddcd81bb538c67e810a3e11696e8bf@gmail.com","name":"Yu ISHIKAWA"},"message":"[MINOR] Format the comment of `translate` at `functions.scala`\n\nAuthor: Yu ISHIKAWA <yuu.ishikawa@gmail.com>\n\nCloses #8265 from yu-iskw/minor-translate-comment.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/a0910315dae88b033e38a1de07f39ca21f6552ad"},{"sha":"5af3838d2e59ed83766f85634e26918baa53819f","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-10038] [SQL] fix bug in generated unsafe projection when there is binary in ArrayData\n\nThe type for array of array in Java is slightly different than array of others.\n\ncc cloud-fan\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #8250 from davies/array_binary.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/5af3838d2e59ed83766f85634e26918baa53819f"},{"sha":"dd0614fd618ad28cb77aecfbd49bb319b98fdba0","author":{"email":"452ae006f36be6c70342f33f64bb6f6eb552689c@gmail.com","name":"Yanbo Liang"},"message":"[SPARK-10076] [ML] make MultilayerPerceptronClassifier layers and weights public\n\nFix the issue that ```layers``` and ```weights``` should be public variables of ```MultilayerPerceptronClassificationModel```. Users can not get ```layers``` and ```weights``` from a ```MultilayerPerceptronClassificationModel``` currently.\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #8263 from yanboliang/mlp-public.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/dd0614fd618ad28cb77aecfbd49bb319b98fdba0"},{"sha":"c34e9ff0eac2032283b959fe63b47cc30f28d21c","author":{"email":"fd726277649d3b2e3ee5d332dca6c89739b68aa5@gmail.com","name":"CodingCat"},"message":"[MINOR] fix the comments in IndexShuffleBlockResolver\n\nit might be a typo  introduced at the first moment or some leftover after some renaming......\n\nthe name of the method accessing the index file is called `getBlockData` now (not `getBlockLocation` as indicated in the comments)\n\nAuthor: CodingCat <zhunansjtu@gmail.com>\n\nCloses #8238 from CodingCat/minor_1.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/c34e9ff0eac2032283b959fe63b47cc30f28d21c"},{"sha":"5723d26d7e677b89383de3fcf2c9a821b68a65b7","author":{"email":"e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","name":"Cheng Lian"},"message":"[SPARK-8118] [SQL] Redirects Parquet JUL logger via SLF4J\n\nParquet hard coded a JUL logger which always writes to stdout. This PR redirects it via SLF4j JUL bridge handler, so that we can control Parquet logs via `log4j.properties`.\n\nThis solution is inspired by https://github.com/Parquet/parquet-mr/issues/390#issuecomment-46064909.\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #8196 from liancheng/spark-8118/redirect-parquet-jul.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/5723d26d7e677b89383de3fcf2c9a821b68a65b7"},{"sha":"1968276af0f681fe51328b7dd795bd21724a5441","author":{"email":"bc4a65f227ddcd81bb538c67e810a3e11696e8bf@gmail.com","name":"Yuu ISHIKAWA"},"message":"[SPARK-10007] [SPARKR] Update `NAMESPACE` file in SparkR for simple parameters functions\n\n### JIRA\n[[SPARK-10007] Update `NAMESPACE` file in SparkR for simple parameters functions - ASF JIRA](https://issues.apache.org/jira/browse/SPARK-10007)\n\nAuthor: Yuu ISHIKAWA <yuu.ishikawa@gmail.com>\n\nCloses #8277 from yu-iskw/SPARK-10007.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/1968276af0f681fe51328b7dd795bd21724a5441"},{"sha":"354f4582b637fa25d3892ec2b12869db50ed83c9","author":{"email":"66332c93c294b372ab1bdd5acb9b89a538e2d2f2@gmail.com","name":"Yuhao Yang"},"message":"[SPARK-9028] [ML] Add CountVectorizer as an estimator to generate CountVectorizerModel\n\njira: https://issues.apache.org/jira/browse/SPARK-9028\n\nAdd an estimator for CountVectorizerModel. The estimator will extract a vocabulary from document collections according to the term frequency.\n\nI changed the meaning of minCount as a filter across the corpus. This aligns with Word2Vec and the similar parameter in SKlearn.\n\nAuthor: Yuhao Yang <hhbyyh@gmail.com>\nAuthor: Joseph K. Bradley <joseph@databricks.com>\n\nCloses #7388 from hhbyyh/cvEstimator.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/354f4582b637fa25d3892ec2b12869db50ed83c9"},{"sha":"c1840a862eb548bc4306e53ee7e9f26986b31832","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-7736] [CORE] Fix a race introduced in PythonRunner.\n\nThe fix for SPARK-7736 introduced a race where a port value of \"-1\"\ncould be passed down to the pyspark process, causing it to fail to\nconnect back to the JVM. This change adds code to fix that race.\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #8258 from vanzin/SPARK-7736.","distinct":false,"url":"https://api.github.com/repos/jkbradley/spark/commits/c1840a862eb548bc4306e53ee7e9f26986b31832"}]}