{"push_id":684635296,"size":25,"distinct_size":0,"ref":"refs/heads/master","head":"9982d453c39e50aedae7d01e4c38fab1b2bc6be0","before":"28dbde3874ccdd44b73675938719b69336d23dac","commits":[{"sha":"f1646e1023bd03e27268a8aa2ea11b6cc284075f","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-7973] [SQL] Increase the timeout of two CliSuite tests.\n\nhttps://issues.apache.org/jira/browse/SPARK-7973\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #6525 from yhuai/SPARK-7973 and squashes the following commits:\n\n763b821 [Yin Huai] Also change the timeout of \"Single command with -e\" to 2 minutes.\ne598a08 [Yin Huai] Increase the timeout to 3 minutes.","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/f1646e1023bd03e27268a8aa2ea11b6cc284075f"},{"sha":"2c4d550eda0e6f33d2d575825c3faef4c9217067","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"[SPARK-7801] [BUILD] Updating versions to SPARK 1.5.0\n\nAuthor: Patrick Wendell <patrick@databricks.com>\n\nCloses #6328 from pwendell/spark-1.5-update and squashes the following commits:\n\n2f42d02 [Patrick Wendell] A few more excludes\n4bebcf0 [Patrick Wendell] Update to RC4\n61aaf46 [Patrick Wendell] Using new release candidate\n55f1610 [Patrick Wendell] Another exclude\n04b4f04 [Patrick Wendell] More issues with transient 1.4 changes\n36f549b [Patrick Wendell] [SPARK-7801] [BUILD] Updating versions to SPARK 1.5.0","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/2c4d550eda0e6f33d2d575825c3faef4c9217067"},{"sha":"d053a31be93d789e3f26cf55d747ecf6ca386c29","author":{"email":"230d029330a575d33cdf1a86f8114e5e5807912e@apache.spark","name":"animesh"},"message":"[SPARK-7980] [SQL] Support SQLContext.range(end)\n\n1. range() overloaded in SQLContext.scala\n2. range() modified in python sql context.py\n3. Tests added accordingly in DataFrameSuite.scala and python sql tests.py\n\nAuthor: animesh <animesh@apache.spark>\n\nCloses #6609 from animeshbaranawal/SPARK-7980 and squashes the following commits:\n\n935899c [animesh] SPARK-7980:python+scala changes","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/d053a31be93d789e3f26cf55d747ecf6ca386c29"},{"sha":"d2a86eb8f0fcc02304604da56c589ea58c77587a","author":{"email":"e1f8d9790f67825b23faf6a373ab7c83bc8900f2@apache.org","name":"Hari Shreedharan"},"message":"[SPARK-7161] [HISTORY SERVER] Provide REST api to download event logs fro...\n\n...m History Server\n\nThis PR adds a new API that allows the user to download event logs for an application as a zip file. APIs have been added to download all logs for a given application or just for a specific attempt.\n\nThis also add an additional method to the ApplicationHistoryProvider to get the raw files, zipped.\n\nAuthor: Hari Shreedharan <hshreedharan@apache.org>\n\nCloses #5792 from harishreedharan/eventlog-download and squashes the following commits:\n\n221cc26 [Hari Shreedharan] Update docs with new API information.\na131be6 [Hari Shreedharan] Fix style issues.\n5528bd8 [Hari Shreedharan] Merge branch 'master' into eventlog-download\n6e8156e [Hari Shreedharan] Simplify tests, use Guava stream copy methods.\nd8ddede [Hari Shreedharan] Remove unnecessary case in EventLogDownloadResource.\nffffb53 [Hari Shreedharan] Changed interface to use zip stream. Added more tests.\n1100b40 [Hari Shreedharan] Ensure that `Path` does not appear in interfaces, by rafactoring interfaces.\n5a5f3e2 [Hari Shreedharan] Fix test ordering issue.\n0b66948 [Hari Shreedharan] Minor formatting/import fixes.\n4fc518c [Hari Shreedharan] Fix rat failures.\na48b91f [Hari Shreedharan] Refactor to make attemptId optional in the API. Also added tests.\n0fc1424 [Hari Shreedharan] File download now works for individual attempts and the entire application.\n350d7e8 [Hari Shreedharan] Merge remote-tracking branch 'asf/master' into eventlog-download\nfd6ab00 [Hari Shreedharan] Fix style issues\n32b7662 [Hari Shreedharan] Use UIRoot directly in ApiRootResource. Also, use `Response` class to set headers.\n7b362b2 [Hari Shreedharan] Almost working.\n3d18ebc [Hari Shreedharan] [WIP] Try getting the event log download to work.","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/d2a86eb8f0fcc02304604da56c589ea58c77587a"},{"sha":"708c63bbbe9580eb774fe47e23ef61338103afda","author":{"email":"3f27eab2afd49386f0ac811afb0bc245aa92c0fa@intel.com","name":"Sun Rui"},"message":"[SPARK-8063] [SPARKR] Spark master URL conflict between MASTER env variable and --master command line option.\n\nAuthor: Sun Rui <rui.sun@intel.com>\n\nCloses #6605 from sun-rui/SPARK-8063 and squashes the following commits:\n\n51ca48b [Sun Rui] [SPARK-8063][SPARKR] Spark master URL conflict between MASTER env variable and --master command line option.","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/708c63bbbe9580eb774fe47e23ef61338103afda"},{"sha":"939e4f3d8def16dfe03f0196be8e1c218a9daa32","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-8074] Parquet should throw AnalysisException during setup for data type/name related failures.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #6608 from rxin/parquet-analysis and squashes the following commits:\n\nb5dc8e2 [Reynold Xin] Code review feedback.\n5617cf6 [Reynold Xin] [SPARK-8074] Parquet should throw AnalysisException during setup for data type/name related failures.","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/939e4f3d8def16dfe03f0196be8e1c218a9daa32"},{"sha":"2c5a06cafd2885ff5431fa96485db2564ae1cce3","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"Update documentation for [SPARK-7980] [SQL] Support SQLContext.range(end)","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/2c5a06cafd2885ff5431fa96485db2564ae1cce3"},{"sha":"20a26b595c74ac41cf7c19e6091d7e675e503321","author":{"email":"461476587780aa9fa5611ea6dc3912c146a91760@databricks.com","name":"Joseph K. Bradley"},"message":"[SPARK-8054] [MLLIB] Added several Java-friendly APIs + unit tests\n\nJava-friendly APIs added:\n* GaussianMixture.run()\n* GaussianMixtureModel.predict()\n* DistributedLDAModel.javaTopicDistributions()\n* StreamingKMeans: trainOn, predictOn, predictOnValues\n* Statistics.corr\n* params\n  * added doc to w() since Java docs do not inherit doc\n  * removed non-Java-friendly w() from StringArrayParam and DoubleArrayParam\n  * made DoubleArrayParam Java-friendly w() actually Java-friendly\n\nI generated the doc and verified all changes.\n\nCC: mengxr\n\nAuthor: Joseph K. Bradley <joseph@databricks.com>\n\nCloses #6562 from jkbradley/java-api-1.4 and squashes the following commits:\n\nc16821b [Joseph K. Bradley] Small fixes based on code review.\nd955581 [Joseph K. Bradley] unit test fixes\n29b6b0d [Joseph K. Bradley] small fixes\nfe6dcfe [Joseph K. Bradley] Added several Java-friendly APIs + unit tests: NaiveBayes, GaussianMixture, LDA, StreamingKMeans, Statistics.corr, params","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/20a26b595c74ac41cf7c19e6091d7e675e503321"},{"sha":"c6a6dd0d0736d548ff9f255e5ed5df45b29c46c1","author":{"email":"02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@databricks.com","name":"Andrew Or"},"message":"[MINOR] [UI] Improve confusing message on log page\n\nIt's good practice to check if the input path is in the directory\nwe expect to avoid potentially confusing error messages.","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/c6a6dd0d0736d548ff9f255e5ed5df45b29c46c1"},{"sha":"bfbf12b349e998c7e674649a07b88c4658ae0711","author":{"email":"22a0dda5f6015b8ee84a30ff6553d35a91fe8732@gmail.com","name":"Timothy Chen"},"message":"[SPARK-8083] [MESOS] Use the correct base path in mesos driver page.\n\nAuthor: Timothy Chen <tnachen@gmail.com>\n\nCloses #6615 from tnachen/mesos_driver_path and squashes the following commits:\n\n4f47b7c [Timothy Chen] Use the correct base path in mesos driver page.","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/bfbf12b349e998c7e674649a07b88c4658ae0711"},{"sha":"aa40c4420717aa06a7964bd30b428fb73548beb2","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-8059] [YARN] Wake up allocation thread when new requests arrive.\n\nThis should help reduce latency for new executor allocations.\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #6600 from vanzin/SPARK-8059 and squashes the following commits:\n\n8387a3a [Marcelo Vanzin] [SPARK-8059] [yarn] Wake up allocation thread when new requests arrive.","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/aa40c4420717aa06a7964bd30b428fb73548beb2"},{"sha":"1d8669f15c136cd81f494dd487400c62c9498602","author":{"email":"5d14e010a5b345b784fd604efcc6cebb79f97468@gmail.com","name":"zsxwing"},"message":"[SPARK-8001] [CORE] Make AsynchronousListenerBus.waitUntilEmpty throw TimeoutException if timeout\n\nSome places forget to call `assert` to check the return value of `AsynchronousListenerBus.waitUntilEmpty`. Instead of adding `assert` in these places, I think it's better to make `AsynchronousListenerBus.waitUntilEmpty` throw `TimeoutException`.\n\nAuthor: zsxwing <zsxwing@gmail.com>\n\nCloses #6550 from zsxwing/SPARK-8001 and squashes the following commits:\n\n607674a [zsxwing] Make AsynchronousListenerBus.waitUntilEmpty throw TimeoutException if timeout","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/1d8669f15c136cd81f494dd487400c62c9498602"},{"sha":"f27134782ebb61c360330e2d6d5bb1aa02be3fb6","author":{"email":"5d14e010a5b345b784fd604efcc6cebb79f97468@gmail.com","name":"zsxwing"},"message":"[SPARK-7989] [CORE] [TESTS] Fix flaky tests in ExternalShuffleServiceSuite and SparkListenerWithClusterSuite\n\nThe flaky tests in ExternalShuffleServiceSuite and SparkListenerWithClusterSuite will fail if there are not enough executors up before running the jobs.\n\nThis PR adds `JobProgressListener.waitUntilExecutorsUp`. The tests for the cluster mode can use it to wait until the expected executors are up.\n\nAuthor: zsxwing <zsxwing@gmail.com>\n\nCloses #6546 from zsxwing/SPARK-7989 and squashes the following commits:\n\n5560e09 [zsxwing] Fix a typo\n3b69840 [zsxwing] Fix flaky tests in ExternalShuffleServiceSuite and SparkListenerWithClusterSuite","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/f27134782ebb61c360330e2d6d5bb1aa02be3fb6"},{"sha":"a8f1f1543e29fb2897e9ae6940581b9e4a3a13fb","author":{"email":"e1f8d9790f67825b23faf6a373ab7c83bc8900f2@apache.org","name":"Hari Shreedharan"},"message":"[HOTFIX] Fix Hadoop-1 build caused by #5792.\n\nReplaced `fs.listFiles` with Hadoop-1 friendly `fs.listStatus` method.\n\nAuthor: Hari Shreedharan <hshreedharan@apache.org>\n\nCloses #6619 from harishreedharan/evetlog-hadoop-1-fix and squashes the following commits:\n\n6192078 [Hari Shreedharan] [HOTFIX] Fix Hadoop-1 build caused by #5972.","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/a8f1f1543e29fb2897e9ae6940581b9e4a3a13fb"},{"sha":"d3e026f8798f9875b90e8c372056ee3d71489be5","author":{"email":"ad63b47dfac1951bde2b661ba9fbbe123f686bee@cs.berkeley.edu","name":"Shivaram Venkataraman"},"message":"[SPARK-3674] [EC2] Clear SPARK_WORKER_INSTANCES when using YARN\n\ncc andrewor14\n\nAuthor: Shivaram Venkataraman <shivaram@cs.berkeley.edu>\n\nCloses #6424 from shivaram/spark-worker-instances-yarn-ec2 and squashes the following commits:\n\ndb244ae [Shivaram Venkataraman] Make Python Lint happy\n0593d1b [Shivaram Venkataraman] Clear SPARK_WORKER_INSTANCES when using YARN","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/d3e026f8798f9875b90e8c372056ee3d71489be5"},{"sha":"26c9d7a0f975009e22ec91e5c0b5cfcada79b35e","author":{"email":"9616cf314f60cbadfd3d5d51110130ad7923defb@databricks.com","name":"Xiangrui Meng"},"message":"[SPARK-8051] [MLLIB] make StringIndexerModel silent if input column does not exist\n\nThis is just a workaround to a bigger problem. Some pipeline stages may not be effective during prediction, and they should not complain about missing required columns, e.g. `StringIndexerModel`. jkbradley\n\nAuthor: Xiangrui Meng <meng@databricks.com>\n\nCloses #6595 from mengxr/SPARK-8051 and squashes the following commits:\n\nb6a36b9 [Xiangrui Meng] add doc\nf143fd4 [Xiangrui Meng] Merge remote-tracking branch 'apache/master' into SPARK-8051\n8ee7c7e [Xiangrui Meng] use SparkFunSuite\ne112394 [Xiangrui Meng] make StringIndexerModel silent if input column does not exist","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/26c9d7a0f975009e22ec91e5c0b5cfcada79b35e"},{"sha":"d8662cd909a41575df6e0ea1630d2386d3711240","author":{"email":"dab2ada7598413bb8872ace161bff6ac41fd2b82@salesforce.com","name":"leahmcguire"},"message":"[SPARK-6164] [ML] CrossValidatorModel should keep stats from fitting\n\nAdded stats from cross validation as a val in the cross validation model to save them for user access.\n\nAuthor: leahmcguire <lmcguire@salesforce.com>\n\nCloses #5915 from leahmcguire/saveCVmetrics and squashes the following commits:\n\n49b507b [leahmcguire] fixed tyle error\n67537b1 [leahmcguire] rebased\n85907f0 [leahmcguire] fixed name\n59987cc [leahmcguire] changed param name and test according to comments\n36e71e3 [leahmcguire] rebasing\n4b8223e [leahmcguire] fixed name\n4ddffc6 [leahmcguire] changed param name and test according to comments\n3a995da [leahmcguire] Added stats from cross validation as a val in the cross validation model to save them for user access","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/d8662cd909a41575df6e0ea1630d2386d3711240"},{"sha":"bfbdab12dd37587e5518dcbb76507b752759cace","author":{"email":"02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@databricks.com","name":"Andrew Or"},"message":"[HOTFIX] [TYPO] Fix typo in #6546","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/bfbdab12dd37587e5518dcbb76507b752759cace"},{"sha":"566cb5947925c79ef90af72346672ab7d27bf4df","author":{"email":"e1f8d9790f67825b23faf6a373ab7c83bc8900f2@apache.org","name":"Hari Shreedharan"},"message":"[HOTFIX] History Server API docs error fix.\n\nMinor error in the monitoring docs. Also made indentation changes in `ApiRootResource`\n\nAuthor: Hari Shreedharan <hshreedharan@apache.org>\n\nCloses #6628 from harishreedharan/eventlog-formatting and squashes the following commits:\n\na12553d [Hari Shreedharan] Javadoc updates.\nca399b6 [Hari Shreedharan] [HOTFIX] History Server API docs error fix.","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/566cb5947925c79ef90af72346672ab7d27bf4df"},{"sha":"51898b5158ac7e7e67b0539bc062c9c16ce9a7ce","author":{"email":"45d0cd84ae16b71953c1beb3be9ef57773a10b25@gmail.com","name":"Ryan Williams"},"message":"[SPARK-8088] don't attempt to lower number of executors by 0\n\nAuthor: Ryan Williams <ryan.blake.williams@gmail.com>\n\nCloses #6624 from ryan-williams/execs and squashes the following commits:\n\nb6f71d4 [Ryan Williams] don't attempt to lower number of executors by 0","distinct":false,"url":"https://api.github.com/repos/sebastian-alfers/spark/commits/51898b5158ac7e7e67b0539bc062c9c16ce9a7ce"}]}