{"push_id":774042692,"size":21,"distinct_size":21,"ref":"refs/heads/master","head":"93db313b7d4881f3571e10fedeb1f448cc4778e7","before":"1d43842452ec40f1d3c40fb03a3748d53d249192","commits":[{"sha":"6185cdd2afcd492b77ff225b477b3624e3bc7bb2","author":{"email":"66332c93c294b372ab1bdd5acb9b89a538e2d2f2@gmail.com","name":"Yuhao Yang"},"message":"[SPARK-9901] User guide for RowMatrix Tall-and-skinny QR\n\njira: https://issues.apache.org/jira/browse/SPARK-9901\n\nThe jira covers only the document update. I can further provide example code for QR (like the ones for SVD and PCA) in a separate PR.\n\nAuthor: Yuhao Yang <hhbyyh@gmail.com>\n\nCloses #8462 from hhbyyh/qrDoc.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/6185cdd2afcd492b77ff225b477b3624e3bc7bb2"},{"sha":"c94ecdfc5b3c0fe6c38a170dc2af9259354dc9e3","author":{"email":"8757c4a255384de9fc2175f17a84f948e7081d30@gmail.com","name":"MechCoder"},"message":"[SPARK-9906] [ML] User guide for LogisticRegressionSummary\n\nUser guide for LogisticRegression summaries\n\nAuthor: MechCoder <manojkumarsivaraj334@gmail.com>\nAuthor: Manoj Kumar <mks542@nyu.edu>\nAuthor: Feynman Liang <fliang@databricks.com>\n\nCloses #8197 from MechCoder/log_summary_user_guide.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/c94ecdfc5b3c0fe6c38a170dc2af9259354dc9e3"},{"sha":"5bfe9e1111d9862084586549a7dc79476f67bab9","author":{"email":"9835e449f9a666ddd8ba8aeb460af39a5130691b@databricks.com","name":"Feynman Liang"},"message":"[SPARK-9680] [MLLIB] [DOC] StopWordsRemovers user guide and Java compatibility test\n\n* Adds user guide for ml.feature.StopWordsRemovers, ran code examples on my machine\n* Cleans up scaladocs for public methods\n* Adds test for Java compatibility\n* Follow up Python user guide code example is tracked by SPARK-10249\n\nAuthor: Feynman Liang <fliang@databricks.com>\n\nCloses #8436 from feynmanliang/SPARK-10230.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/5bfe9e1111d9862084586549a7dc79476f67bab9"},{"sha":"b3dd569ad40905f8861a547a1e25ed3ca8e1d272","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-10287] [SQL] Fixes JSONRelation refreshing on read path\n\nhttps://issues.apache.org/jira/browse/SPARK-10287\n\nAfter porting json to HadoopFsRelation, it seems hard to keep the behavior of picking up new files automatically for JSON. This PR removes this behavior, so JSON is consistent with others (ORC and Parquet).\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #8469 from yhuai/jsonRefresh.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/b3dd569ad40905f8861a547a1e25ed3ca8e1d272"},{"sha":"54cda0deb6bebf1470f16ba5bcc6c4fb842bdac1","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-10321] sizeInBytes in HadoopFsRelation\n\nHaving sizeInBytes in HadoopFsRelation to enable broadcast join.\n\ncc marmbrus\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #8490 from davies/sizeInByte.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/54cda0deb6bebf1470f16ba5bcc6c4fb842bdac1"},{"sha":"1f90c5e2198bcf49e115d97ec300c17c1be4dcb4","author":{"email":"bc4a65f227ddcd81bb538c67e810a3e11696e8bf@gmail.com","name":"Yu ISHIKAWA"},"message":"[SPARK-8505] [SPARKR] Add settings to kick `lint-r` from `./dev/run-test.py`\n\nJoshRosen we'd like to check the SparkR source code with the `dev/lint-r` script on the Jenkins. I tried to incorporate the script into `dev/run-test.py`. Could you review it when you have time?\n\nshivaram I modified `dev/lint-r` and `dev/lint-r.R` to install lintr package into a local directory(`R/lib/`) and to exit with a lint status. Could you review it?\n\n- [[SPARK-8505] Add settings to kick `lint-r` from `./dev/run-test.py` - ASF JIRA](https://issues.apache.org/jira/browse/SPARK-8505)\n\nAuthor: Yu ISHIKAWA <yuu.ishikawa@gmail.com>\n\nCloses #7883 from yu-iskw/SPARK-8505.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/1f90c5e2198bcf49e115d97ec300c17c1be4dcb4"},{"sha":"30734d45fbbb269437c062241a9161e198805a76","author":{"email":"8757c4a255384de9fc2175f17a84f948e7081d30@gmail.com","name":"MechCoder"},"message":"[SPARK-9911] [DOC] [ML] Update Userguide for Evaluator\n\nI added a small note about the different types of evaluator and the metrics used.\n\nAuthor: MechCoder <manojkumarsivaraj334@gmail.com>\n\nCloses #8304 from MechCoder/multiclass_evaluator.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/30734d45fbbb269437c062241a9161e198805a76"},{"sha":"af0e1249b1c881c0fa7a921fd21fd2c27214b980","author":{"email":"9835e449f9a666ddd8ba8aeb460af39a5130691b@databricks.com","name":"Feynman Liang"},"message":"[SPARK-9905] [ML] [DOC] Adds LinearRegressionSummary user guide\n\n* Adds user guide for `LinearRegressionSummary`\n* Fixes unresolved issues in  #8197\n\nCC jkbradley mengxr\n\nAuthor: Feynman Liang <fliang@databricks.com>\n\nCloses #8491 from feynmanliang/SPARK-9905.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/af0e1249b1c881c0fa7a921fd21fd2c27214b980"},{"sha":"89b943438512fcfb239c268b43431397de46cbcf","author":{"email":"e6f14efef9fae0ae8f92d04007faea68035873ce@databricks.com","name":"Cheng Lian"},"message":"[SPARK-SQL] [MINOR] Fixes some typos in HiveContext\n\nAuthor: Cheng Lian <lian@databricks.com>\n\nCloses #8481 from liancheng/hive-context-typo.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/89b943438512fcfb239c268b43431397de46cbcf"},{"sha":"7583681e6b0824d7eed471dc4d8fa0b2addf9ffc","author":{"email":"1d6e1cf70ec6f9ab28d3ea4b27a49a77654d370e@noelsmith.com","name":"noelsmith"},"message":"[SPARK-10188] [PYSPARK] Pyspark CrossValidator with RMSE selects incorrect model\n\n* Added isLargerBetter() method to Pyspark Evaluator to match the Scala version.\n* JavaEvaluator delegates isLargerBetter() to underlying Scala object.\n* Added check for isLargerBetter() in CrossValidator to determine whether to use argmin or argmax.\n* Added test cases for where smaller is better (RMSE) and larger is better (R-Squared).\n\n(This contribution is my original work and that I license the work to the project under Sparks' open source license)\n\nAuthor: noelsmith <mail@noelsmith.com>\n\nCloses #8399 from noel-smith/pyspark-rmse-xval-fix.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/7583681e6b0824d7eed471dc4d8fa0b2addf9ffc"},{"sha":"2f99c37273c1d82e2ba39476e4429ea4aaba7ec6","author":{"email":"ad63b47dfac1951bde2b661ba9fbbe123f686bee@cs.berkeley.edu","name":"Shivaram Venkataraman"},"message":"[SPARK-10328] [SPARKR] Fix generic for na.omit\n\nS3 function is at https://stat.ethz.ch/R-manual/R-patched/library/stats/html/na.fail.html\n\nAuthor: Shivaram Venkataraman <shivaram@cs.berkeley.edu>\nAuthor: Shivaram Venkataraman <shivaram.venkataraman@gmail.com>\nAuthor: Yu ISHIKAWA <yuu.ishikawa@gmail.com>\n\nCloses #8495 from shivaram/na-omit-fix.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/2f99c37273c1d82e2ba39476e4429ea4aaba7ec6"},{"sha":"4eeda8d472498acd40ef54723d1be9924a273d76","author":{"email":"bc4a65f227ddcd81bb538c67e810a3e11696e8bf@gmail.com","name":"Yu ISHIKAWA"},"message":"[SPARK-10260] [ML] Add @Since annotation to ml.clustering\n\n### JIRA\n[[SPARK-10260] Add Since annotation to ml.clustering - ASF JIRA](https://issues.apache.org/jira/browse/SPARK-10260)\n\nAuthor: Yu ISHIKAWA <yuu.ishikawa@gmail.com>\n\nCloses #8455 from yu-iskw/SPARK-10260.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/4eeda8d472498acd40ef54723d1be9924a273d76"},{"sha":"cc39803062119c1d14611dc227b9ed0ed1284d38","author":{"email":"291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","name":"Sean Owen"},"message":"[SPARK-10295] [CORE] Dynamic allocation in Mesos does not release when RDDs are cached\n\nRemove obsolete warning about dynamic allocation not working with cached RDDs\n\nSee discussion in https://issues.apache.org/jira/browse/SPARK-10295\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #8489 from srowen/SPARK-10295.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/cc39803062119c1d14611dc227b9ed0ed1284d38"},{"sha":"18294cd8710427076caa86bfac596de67089d57e","author":{"email":"38bfc72e6a35a427ed65527be3b49f08e9586edf@gmail.com","name":"Keiji Yoshida"},"message":"Fix DynamodDB/DynamoDB typo in Kinesis Integration doc\n\nFix DynamodDB/DynamoDB typo in Kinesis Integration doc\n\nAuthor: Keiji Yoshida <yoshida.keiji.84@gmail.com>\n\nCloses #8501 from yosssi/patch-1.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/18294cd8710427076caa86bfac596de67089d57e"},{"sha":"71a077f6c16c8816eae13341f645ba50d997f63d","author":{"email":"774f39a2892fba124fafe3b4d4e84a5d765e8ade@users.noreply.github.com","name":"Dharmesh Kakadia"},"message":"typo in comment\n\nAuthor: Dharmesh Kakadia <dharmeshkakadia@users.noreply.github.com>\n\nCloses #8497 from dharmeshkakadia/patch-2.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/71a077f6c16c8816eae13341f645ba50d997f63d"},{"sha":"1502a0f6c5d2f85a331b29d3bf50002911ea393e","author":{"email":"2676c1e0be9d9df57419d05258d46b7142312de2@hortonworks.com","name":"jerryshao"},"message":"[YARN] [MINOR] Avoid hard code port number in YarnShuffleService test\n\nCurrent port number is fixed as default (7337) in test, this will introduce port contention exception, better to change to a random number in unit test.\n\nsquito , seems you're author of this unit test, mind taking a look at this fix? Thanks a lot.\n\n```\n[info] - executor state kept across NM restart *** FAILED *** (597 milliseconds)\n[info]   org.apache.hadoop.service.ServiceStateException: java.net.BindException: Address already in use\n[info]   at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)\n[info]   at org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)\n[info]   at org.apache.spark.network.yarn.YarnShuffleServiceSuite$$anonfun$1.apply$mcV$sp(YarnShuffleServiceSuite.scala:72)\n[info]   at org.apache.spark.network.yarn.YarnShuffleServiceSuite$$anonfun$1.apply(YarnShuffleServiceSuite.scala:70)\n[info]   at org.apache.spark.network.yarn.YarnShuffleServiceSuite$$anonfun$1.apply(YarnShuffleServiceSuite.scala:70)\n[info]   at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)\n[info]   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)\n[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)\n[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)\n[info]   at org.scalatest.Transformer.apply(Transformer.scala:20)\n[info]   at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)\n[info]   at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:42)\n...\n```\n\nAuthor: jerryshao <sshao@hortonworks.com>\n\nCloses #8502 from jerryshao/avoid-hardcode-port.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/1502a0f6c5d2f85a331b29d3bf50002911ea393e"},{"sha":"e2a843090cb031f6aa774f6d9c031a7f0f732ee1","author":{"email":"66332c93c294b372ab1bdd5acb9b89a538e2d2f2@gmail.com","name":"Yuhao Yang"},"message":"[SPARK-9890] [DOC] [ML] User guide for CountVectorizer\n\njira: https://issues.apache.org/jira/browse/SPARK-9890\n\ndocument with Scala and java examples\n\nAuthor: Yuhao Yang <hhbyyh@gmail.com>\n\nCloses #8487 from hhbyyh/cvDoc.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/e2a843090cb031f6aa774f6d9c031a7f0f732ee1"},{"sha":"499e8e154bdcc9d7b2f685b159e0ddb4eae48fe4","author":{"email":"00d7a42e7227da37af994b379e3eb0eed5cbdaa7@apache.org","name":"Luciano Resende"},"message":"[SPARK-8952] [SPARKR] - Wrap normalizePath calls with suppressWarnings\n\nThis is based on davies comment on SPARK-8952 which suggests to only call normalizePath() when path starts with '~'\n\nAuthor: Luciano Resende <lresende@apache.org>\n\nCloses #8343 from lresende/SPARK-8952.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/499e8e154bdcc9d7b2f685b159e0ddb4eae48fe4"},{"sha":"d3f87dc39480f075170817bbd00142967a938078","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-10325] Override hashCode() for public Row\n\nThis commit fixes an issue where the public SQL `Row` class did not override `hashCode`, causing it to violate the hashCode() + equals() contract. To fix this, I simply ported the `hashCode` implementation from the 1.4.x version of `Row`.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #8500 from JoshRosen/SPARK-10325 and squashes the following commits:\n\n51ffea1 [Josh Rosen] Override hashCode() for public Row.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/d3f87dc39480f075170817bbd00142967a938078"},{"sha":"c53c902fa9c458200245f919067b41dde9cd9418","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-9284] [TESTS] Allow all tests to run without an assembly.\n\nThis change aims at speeding up the dev cycle a little bit, by making\nsure that all tests behave the same w.r.t. where the code to be tested\nis loaded from. Namely, that means that tests don't rely on the assembly\nanymore, rather loading all needed classes from the build directories.\n\nThe main change is to make sure all build directories (classes and test-classes)\nare added to the classpath of child processes when running tests.\n\nYarnClusterSuite required some custom code since the executors are run\ndifferently (i.e. not through the launcher library, like standalone and\nMesos do).\n\nI also found a couple of tests that could leak a SparkContext on failure,\nand added code to handle those.\n\nWith this patch, it's possible to run the following command from a clean\nsource directory and have all tests pass:\n\n  mvn -Pyarn -Phadoop-2.4 -Phive-thriftserver install\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #7629 from vanzin/SPARK-9284.","distinct":true,"url":"https://api.github.com/repos/hxfeng/spark/commits/c53c902fa9c458200245f919067b41dde9cd9418"}]}