{"push_id":538741034,"size":30,"distinct_size":30,"ref":"refs/heads/master","head":"d3f07fd23cc26a70f44c52e24445974d4885d58a","before":"8f29b7cafc2b6e802e4eb21f681d6369da2f30fa","commits":[{"sha":"07fa1910d9c4092d670381c447403105f01c584e","author":{"email":"9eee3e63534349a30e59f0f71d1c0ae509ca1e91@gmail.com","name":"wangxiaojing"},"message":"[SPARK-4570][SQL]add BroadcastLeftSemiJoinHash\n\nJIRA issue: [SPARK-4570](https://issues.apache.org/jira/browse/SPARK-4570)\nWe are planning to create a `BroadcastLeftSemiJoinHash` to implement the broadcast join for `left semijoin`\nIn left semijoin :\nIf the size of data from right side is smaller than the user-settable threshold `AUTO_BROADCASTJOIN_THRESHOLD`,\nthe planner would mark it as the `broadcast` relation and mark the other relation as the stream side. The broadcast table will be broadcasted to all of the executors involved in the join, as a `org.apache.spark.broadcast.Broadcast` object. It will use `joins.BroadcastLeftSemiJoinHash`.,else it will use `joins.LeftSemiJoinHash`.\n\nThe benchmark suggests these  made the optimized version 4x faster  when `left semijoin`\n<pre><code>\nOriginal:\nleft semi join : 9288 ms\nOptimized:\nleft semi join : 1963 ms\n</code></pre>\nThe micro benchmark load `data1/kv3.txt` into a normal Hive table.\nBenchmark code:\n<pre><code>\n def benchmark(f: => Unit) = {\n    val begin = System.currentTimeMillis()\n    f\n    val end = System.currentTimeMillis()\n    end - begin\n  }\n  val sc = new SparkContext(\n    new SparkConf()\n      .setMaster(\"local\")\n      .setAppName(getClass.getSimpleName.stripSuffix(\"$\")))\n  val hiveContext = new HiveContext(sc)\n  import hiveContext._\n  sql(\"drop table if exists left_table\")\n  sql(\"drop table if exists right_table\")\n  sql( \"\"\"create table left_table (key int, value string)\n       \"\"\".stripMargin)\n  sql( s\"\"\"load data local inpath \"/data1/kv3.txt\" into table left_table\"\"\")\n  sql( \"\"\"create table right_table (key int, value string)\n       \"\"\".stripMargin)\n  sql(\n    \"\"\"\n      |from left_table\n      |insert overwrite table right_table\n      |select left_table.key, left_table.value\n    \"\"\".stripMargin)\n\n  val leftSimeJoin = sql(\n    \"\"\"select a.key from left_table a\n      |left semi join right_table b on a.key = b.key\"\"\".stripMargin)\n  val leftSemiJoinDuration = benchmark(leftSimeJoin.count())\n  println(s\"left semi join : $leftSemiJoinDuration ms \")\n</code></pre>\n\nAuthor: wangxiaojing <u9jing@gmail.com>\n\nCloses #3442 from wangxiaojing/SPARK-4570 and squashes the following commits:\n\na4a43c9 [wangxiaojing] rebase\nf103983 [wangxiaojing] change style\nfbe4887 [wangxiaojing] change style\nff2e618 [wangxiaojing] add testsuite\n1a8da2a [wangxiaojing] add BroadcastLeftSemiJoinHash","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/07fa1910d9c4092d670381c447403105f01c584e"},{"sha":"b239ea1c31aeaa752d5dc8f45423df1f8c0924ca","author":{"email":"291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","name":"Sean Owen"},"message":"SPARK-3955 part 2 [CORE] [HOTFIX] Different versions between jackson-mapper-asl and jackson-core-asl\n\npwendell https://github.com/apache/spark/commit/2483c1efb6429a7d8a20c96d18ce2fec93a1aff9 didn't actually add a reference to `jackson-core-asl` as intended, but a second redundant reference to `jackson-mapper-asl`, as markhamstra picked up on (https://github.com/apache/spark/pull/3716#issuecomment-68180192)  This just rectifies the typo. I missed it as well; the original PR https://github.com/apache/spark/pull/2818 had it correct and I also didn't see the problem.\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #3829 from srowen/SPARK-3955 and squashes the following commits:\n\n6cfdc4e [Sean Owen] Actually refer to jackson-core-asl","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/b239ea1c31aeaa752d5dc8f45423df1f8c0924ca"},{"sha":"0f31992c61f6662e5347745f6a1ac272a5fd63c9","author":{"email":"6815369a2a24e137459095f505ca9d62a23a73bb@avast.com","name":"Jakub Dubovsky"},"message":"[Spark-4995] Replace Vector.toBreeze.activeIterator with foreachActive\n\nNew foreachActive method of vector was introduced by SPARK-4431 as more efficient alternative to vector.toBreeze.activeIterator. There are some parts of codebase where it was not yet replaced.\n\ndbtsai\n\nAuthor: Jakub Dubovsky <dubovsky@avast.com>\n\nCloses #3846 from james64/SPARK-4995-foreachActive and squashes the following commits:\n\n3eb7e37 [Jakub Dubovsky] Scalastyle fix\n32fe6c6 [Jakub Dubovsky] activeIterator removed - IndexedRowMatrix.toBreeze\n47a4777 [Jakub Dubovsky] activeIterator removed in RowMatrix.toBreeze\n90a7d98 [Jakub Dubovsky] activeIterator removed in MLUtils.saveAsLibSVMFile","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/0f31992c61f6662e5347745f6a1ac272a5fd63c9"},{"sha":"6a897829444e2ef273586511f93a40d36e64fb0b","author":{"email":"5d14e010a5b345b784fd604efcc6cebb79f97468@gmail.com","name":"zsxwing"},"message":"[SPARK-4813][Streaming] Fix the issue that ContextWaiter didn't handle 'spurious wakeup'\n\nUsed `Condition` to rewrite `ContextWaiter` because it provides a convenient API `awaitNanos` for timeout.\n\nAuthor: zsxwing <zsxwing@gmail.com>\n\nCloses #3661 from zsxwing/SPARK-4813 and squashes the following commits:\n\n52247f5 [zsxwing] Add explicit unit type\nbe42bcf [zsxwing] Update as per review suggestion\ne06bd4f [zsxwing] Fix the issue that ContextWaiter didn't handle 'spurious wakeup'","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/6a897829444e2ef273586511f93a40d36e64fb0b"},{"sha":"035bac88c732247c79a1bbad4f9191090cbbdc9a","author":{"email":"dd51fce4c161499974ebc82f8b8e07d60438e3ec@163.com","name":"Liu Jiongzhou"},"message":"[SPARK-4998][MLlib]delete the \"train\" function\n\nTo make the functions with the same in \"object\" effective, specially when using java reflection.\nAs the \"train\" function defined in \"class DecisionTree\" will hide the functions with the same name in \"object DecisionTree\".\n\nJIRA[SPARK-4998]\n\nAuthor: Liu Jiongzhou <ljzzju@163.com>\n\nCloses #3836 from ljzzju/master and squashes the following commits:\n\n4e13133 [Liu Jiongzhou] [MLlib]delete the \"train\" function","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/035bac88c732247c79a1bbad4f9191090cbbdc9a"},{"sha":"352ed6bbe3c3b67e52e298e7c535ae414d96beca","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-1010] Clean up uses of System.setProperty in unit tests\n\nSeveral of our tests call System.setProperty (or test code which implicitly sets system properties) and don't always reset/clear the modified properties, which can create ordering dependencies between tests and cause hard-to-diagnose failures.\n\nThis patch removes most uses of System.setProperty from our tests, since in most cases we can use SparkConf to set these configurations (there are a few exceptions, including the tests of SparkConf itself).\n\nFor the cases where we continue to use System.setProperty, this patch introduces a `ResetSystemProperties` ScalaTest mixin class which snapshots the system properties before individual tests and to automatically restores them on test completion / failure.  See the block comment at the top of the ResetSystemProperties class for more details.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #3739 from JoshRosen/cleanup-system-properties-in-tests and squashes the following commits:\n\n0236d66 [Josh Rosen] Replace setProperty uses in two example programs / tools\n3888fe3 [Josh Rosen] Remove setProperty use in LocalJavaStreamingContext\n4f4031d [Josh Rosen] Add note on why SparkSubmitSuite needs ResetSystemProperties\n4742a5b [Josh Rosen] Clarify ResetSystemProperties trait inheritance ordering.\n0eaf0b6 [Josh Rosen] Remove setProperty call in TaskResultGetterSuite.\n7a3d224 [Josh Rosen] Fix trait ordering\n3fdb554 [Josh Rosen] Remove setProperty call in TaskSchedulerImplSuite\nbee20df [Josh Rosen] Remove setProperty calls in SparkContextSchedulerCreationSuite\n655587c [Josh Rosen] Remove setProperty calls in JobCancellationSuite\n3f2f955 [Josh Rosen] Remove System.setProperty calls in DistributedSuite\ncfe9cce [Josh Rosen] Remove use of system properties in SparkContextSuite\n8783ab0 [Josh Rosen] Remove TestUtils.setSystemProperty, since it is subsumed by the ResetSystemProperties trait.\n633a84a [Josh Rosen] Remove use of system properties in FileServerSuite\n25bfce2 [Josh Rosen] Use ResetSystemProperties in UtilsSuite\n1d1aa5a [Josh Rosen] Use ResetSystemProperties in SizeEstimatorSuite\ndd9492b [Josh Rosen] Use ResetSystemProperties in AkkaUtilsSuite\nb0daff2 [Josh Rosen] Use ResetSystemProperties in BlockManagerSuite\ne9ded62 [Josh Rosen] Use ResetSystemProperties in TaskSchedulerImplSuite\n5b3cb54 [Josh Rosen] Use ResetSystemProperties in SparkListenerSuite\n0995c4b [Josh Rosen] Use ResetSystemProperties in SparkContextSchedulerCreationSuite\nc83ded8 [Josh Rosen] Use ResetSystemProperties in SparkConfSuite\n51aa870 [Josh Rosen] Use withSystemProperty in ShuffleSuite\n60a63a1 [Josh Rosen] Use ResetSystemProperties in JobCancellationSuite\n14a92e4 [Josh Rosen] Use withSystemProperty in FileServerSuite\n628f46c [Josh Rosen] Use ResetSystemProperties in DistributedSuite\n9e3e0dd [Josh Rosen] Add ResetSystemProperties test fixture mixin; use it in SparkSubmitSuite.\n4dcea38 [Josh Rosen] Move withSystemProperty to TestUtils class.","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/352ed6bbe3c3b67e52e298e7c535ae414d96beca"},{"sha":"06a9aa589c518a40a3c7cc201e89d75af77ab93e","author":{"email":"d24118b4297e1ecf0c990d6935f5f9e6e4fd7701@gmail.com","name":"Liang-Chi Hsieh"},"message":"[SPARK-4797] Replace breezeSquaredDistance\n\nThis PR replaces slow breezeSquaredDistance.\n\nAuthor: Liang-Chi Hsieh <viirya@gmail.com>\n\nCloses #3643 from viirya/faster_squareddistance and squashes the following commits:\n\nf28b275 [Liang-Chi Hsieh] Move the implementation to linalg.Vectors and rename as sqdist.\n0bc48ee [Liang-Chi Hsieh] Merge branch 'master' into faster_squareddistance\nba34422 [Liang-Chi Hsieh] Fix bug.\n91849d0 [Liang-Chi Hsieh] Modified for comment.\n44a65ad [Liang-Chi Hsieh] Modified for comments.\n35db395 [Liang-Chi Hsieh] Fix bug and some modifications for comments.\nf4f5ebb [Liang-Chi Hsieh] Follow BLAS.dot pattern to replace intersect, diff with while-loop.\na36e09f [Liang-Chi Hsieh] Use while-loop to replace foreach for better performance.\nd3e0628 [Liang-Chi Hsieh] Make the methods private.\ndd415bc [Liang-Chi Hsieh] Consider different cases of SparseVector and DenseVector.\n13669db [Liang-Chi Hsieh] Replace breezeSquaredDistance.","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/06a9aa589c518a40a3c7cc201e89d75af77ab93e"},{"sha":"8e14c5eb551ab06c94859c7f6d8c6b62b4d00d59","author":{"email":"eaca3ca4b51928dd768a043ece15380fadcb1b95@capitalone.com","name":"Brennon York"},"message":"[SPARK-4298][Core] - The spark-submit cannot read Main-Class from Manifest.\n\nResolves a bug where the `Main-Class` from a .jar file wasn't being read in properly. This was caused by the fact that the `primaryResource` object was a URI and needed to be normalized through a call to `.getPath` before it could be passed into the `JarFile` object.\n\nAuthor: Brennon York <brennon.york@capitalone.com>\n\nCloses #3561 from brennonyork/SPARK-4298 and squashes the following commits:\n\n5e0fce1 [Brennon York] Use string interpolation for error messages, moved comment line from original code to above its necessary code segment\n14daa20 [Brennon York] pushed mainClass assignment into match statement, removed spurious spaces, removed { } from case statements, removed return values\nc6dad68 [Brennon York] Set case statement to support multiple jar URI's and enabled the 'file' URI to load the main-class\n8d20936 [Brennon York] updated to reset the error message back to the default\na043039 [Brennon York] updated to split the uri and jar vals\n8da7cbf [Brennon York] fixes SPARK-4298","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/8e14c5eb551ab06c94859c7f6d8c6b62b4d00d59"},{"sha":"3d194cc75761fceba77b2c91291b36479b8b556c","author":{"email":"291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","name":"Sean Owen"},"message":"SPARK-4547 [MLLIB] OOM when making bins in BinaryClassificationMetrics\n\nNow that I've implemented the basics here, I'm less convinced there is a need for this change, somehow. Callers can downsample before or after. Really the OOM is not in the ROC curve code, but in code that might `collect()` it for local analysis. Still, might be useful to down-sample since the ROC curve probably never needs millions of points.\n\nThis is a first pass. Since the `(score,label)` are already grouped and sorted, I think it's sufficient to just take every Nth such pair, in order to downsample by a factor of N? this is just like retaining every Nth point on the curve, which I think is the goal. All of the data is still used to build the curve of course.\n\nWhat do you think about the API, and usefulness?\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #3702 from srowen/SPARK-4547 and squashes the following commits:\n\n1d34d05 [Sean Owen] Indent and reorganize numBins scaladoc\n692d825 [Sean Owen] Change handling of large numBins, make 2nd consturctor instead of optional param, style change\na03610e [Sean Owen] Add downsamplingFactor to BinaryClassificationMetrics","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/3d194cc75761fceba77b2c91291b36479b8b556c"},{"sha":"e24d3a9a29962023cc722896a14c7bfe06e8e601","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[HOTFIX] Disable Spark UI in SparkSubmitSuite tests\n\nThis should fix a major cause of build breaks when running many parallel tests.","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/e24d3a9a29962023cc722896a14c7bfe06e8e601"},{"sha":"c88a3d7fca20d36ee566d48e0cb91fe33a7a6d99","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-5038][SQL] Add explicit return type for implicit functions in Spark SQL\n\nAs we learned in https://github.com/apache/spark/pull/3580, not explicitly typing implicit functions can lead to compiler bugs and potentially unexpected runtime behavior.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #3859 from rxin/sql-implicits and squashes the following commits:\n\n30c2c24 [Reynold Xin] [SPARK-5038] Add explicit return type for implicit functions in Spark SQL.","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/c88a3d7fca20d36ee566d48e0cb91fe33a7a6d99"},{"sha":"3610d3c615112faef98d94f04efaea602cc4aa8f","author":{"email":"e1f8d9790f67825b23faf6a373ab7c83bc8900f2@apache.org","name":"Hari Shreedharan"},"message":"[SPARK-4790][STREAMING] Fix ReceivedBlockTrackerSuite waits for old file...\n\n...s to get deleted before continuing.\n\nSince the deletes are happening asynchronously, the getFileStatus call might throw an exception in older HDFS\nversions, if the delete happens between the time listFiles is called on the directory and getFileStatus is called\non the file in the getFileStatus method.\n\nThis PR addresses this by adding an option to delete the files synchronously and then waiting for the deletion to\ncomplete before proceeding.\n\nAuthor: Hari Shreedharan <hshreedharan@apache.org>\n\nCloses #3726 from harishreedharan/spark-4790 and squashes the following commits:\n\nbbbacd1 [Hari Shreedharan] Call cleanUpOldLogs only once in the tests.\n3255f17 [Hari Shreedharan] Add test for async deletion. Remove method from ReceiverTracker that does not take waitForCompletion.\ne4c83ec [Hari Shreedharan] Making waitForCompletion a mandatory param. Remove eventually from WALSuite since the cleanup method returns only after all files are deleted.\naf00fd1 [Hari Shreedharan] [SPARK-4790][STREAMING] Fix ReceivedBlockTrackerSuite waits for old files to get deleted before continuing.","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/3610d3c615112faef98d94f04efaea602cc4aa8f"},{"sha":"fdc2aa4918fd4c510f04812b782cc0bfef9a2107","author":{"email":"79478dbbc8641d1d6545f348f655464aefbda99b@intel.com","name":"jerryshao"},"message":"[SPARK-5028][Streaming]Add total received and processed records metrics to Streaming UI\n\nThis is a follow-up work of [SPARK-4537](https://issues.apache.org/jira/browse/SPARK-4537). Adding total received records and processed records metrics back to UI.\n\n![screenshot](https://dl.dropboxusercontent.com/u/19230832/screenshot.png)\n\nAuthor: jerryshao <saisai.shao@intel.com>\n\nCloses #3852 from jerryshao/SPARK-5028 and squashes the following commits:\n\nc8c4877 [jerryshao] Add total received and processed metrics to Streaming UI","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/fdc2aa4918fd4c510f04812b782cc0bfef9a2107"},{"sha":"c4f0b4f334f7f3565375921fcac184ad5b1fb207","author":{"email":"76654c02f3c3f38f074e5665b86fafb46a77566d@columbia.edu","name":"Travis Galoppo"},"message":"SPARK-5020 [MLlib] GaussianMixtureModel.predictMembership() should take an RDD only\n\nRemoved unnecessary parameters to predictMembership()\n\nCC: jkbradley\n\nAuthor: Travis Galoppo <tjg2107@columbia.edu>\n\nCloses #3854 from tgaloppo/spark-5020 and squashes the following commits:\n\n1bf4669 [Travis Galoppo] renamed predictMembership() to predictSoft()\n0f1d96e [Travis Galoppo] SPARK-5020 - Removed superfluous parameters from predictMembership()","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/c4f0b4f334f7f3565375921fcac184ad5b1fb207"},{"sha":"fe6efacc0b865e9e827a1565877077000e63976e","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-5035] [Streaming] ReceiverMessage trait should extend Serializable\n\nSpark Streaming's ReceiverMessage trait should extend Serializable in order to fix a subtle bug that only occurs when running on a real cluster:\n\nIf you attempt to send a fire-and-forget message to a remote Akka actor and that message cannot be serialized, then this seems to lead to more-or-less silent failures. As an optimization, Akka skips message serialization for messages sent within the same JVM. As a result, Spark's unit tests will never fail due to non-serializable Akka messages, but these will cause mostly-silent failures when running on a real cluster.\n\nBefore this patch, here was the code for ReceiverMessage:\n\n```\n/** Messages sent to the NetworkReceiver. */\nprivate[streaming] sealed trait ReceiverMessage\nprivate[streaming] object StopReceiver extends ReceiverMessage\n```\n\nSince ReceiverMessage does not extend Serializable and StopReceiver is a regular `object`, not a `case object`, StopReceiver will throw serialization errors. As a result, graceful receiver shutdown is broken on real clusters (and local-cluster mode) but works in local modes. If you want to reproduce this, try running the word count example from the Streaming Programming Guide in the Spark shell:\n\n```\nimport org.apache.spark._\nimport org.apache.spark.streaming._\nimport org.apache.spark.streaming.StreamingContext._\nval ssc = new StreamingContext(sc, Seconds(10))\n// Create a DStream that will connect to hostname:port, like localhost:9999\nval lines = ssc.socketTextStream(\"localhost\", 9999)\n// Split each line into words\nval words = lines.flatMap(_.split(\" \"))\nimport org.apache.spark.streaming.StreamingContext._\n// Count each word in each batch\nval pairs = words.map(word => (word, 1))\nval wordCounts = pairs.reduceByKey(_ + _)\n// Print the first ten elements of each RDD generated in this DStream to the console\nwordCounts.print()\nssc.start()\nThread.sleep(10000)\nssc.stop(true, true)\n```\n\nPrior to this patch, this would work correctly in local mode but fail when running against a real cluster (it would report that some receivers were not shut down).\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #3857 from JoshRosen/SPARK-5035 and squashes the following commits:\n\n71d0eae [Josh Rosen] [SPARK-5035] ReceiverMessage trait should extend Serializable.","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/fe6efacc0b865e9e827a1565877077000e63976e"},{"sha":"4bb12488d56ea651c56d9688996b464b99095582","author":{"email":"291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","name":"Sean Owen"},"message":"SPARK-2757 [BUILD] [STREAMING] Add Mima test for Spark Sink after 1.10 is released\n\nRe-enable MiMa for Streaming Flume Sink module, now that 1.1.0 is released, per the JIRA TO-DO. That's pretty much all there is to this.\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #3842 from srowen/SPARK-2757 and squashes the following commits:\n\n50ff80e [Sean Owen] Exclude apparent false positive turned up by re-enabling MiMa checks for Streaming Flume Sink\n0e5ba5c [Sean Owen] Re-enable MiMa for Streaming Flume Sink module","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/4bb12488d56ea651c56d9688996b464b99095582"},{"sha":"7749dd6c36a182478b20f4636734c8db0b7ddb00","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-5038] Add explicit return type for implicit functions.\n\nAs we learned in #3580, not explicitly typing implicit functions can lead to compiler bugs and potentially unexpected runtime behavior.\n\nThis is a follow up PR for rest of Spark (outside Spark SQL). The original PR for Spark SQL can be found at https://github.com/apache/spark/pull/3859\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #3860 from rxin/implicit and squashes the following commits:\n\n73702f9 [Reynold Xin] [SPARK-5038] Add explicit return type for implicit functions.","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/7749dd6c36a182478b20f4636734c8db0b7ddb00"},{"sha":"012839807c3dc6e7c8c41ac6e956d52a550bb031","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[HOTFIX] Bind web UI to ephemeral port in DriverSuite\n\nThe job launched by DriverSuite should bind the web UI to an ephemeral port, since it looks like port contention in this test has caused a large number of Jenkins failures when many builds are started simultaneously.  Our tests already disable the web UI, but this doesn't affect subprocesses launched by our tests.  In this case, I've opted to bind to an ephemeral port instead of disabling the UI because disabling features in this test may mask its ability to catch certain bugs.\n\nSee also: e24d3a9\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #3873 from JoshRosen/driversuite-webui-port and squashes the following commits:\n\n48cd05c [Josh Rosen] [HOTFIX] Bind web UI to ephemeral port in DriverSuite.","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/012839807c3dc6e7c8c41ac6e956d52a550bb031"},{"sha":"bd88b7185358ae60efc83dc6cbb3fb1d2bff6074","author":{"email":"88ec0e7df70fbb4dfef58da07dfeb1a4a708d199@gmail.com","name":"Yadong Qi"},"message":"[SPARK-3325][Streaming] Add a parameter to the method print in class DStream\n\nThis PR is a fixed version of the original PR #3237 by watermen and scwf.\nThis adds the ability to specify how many elements to print in `DStream.print`.\n\nAuthor: Yadong Qi <qiyadong2010@gmail.com>\nAuthor: q00251598 <qiyadong@huawei.com>\nAuthor: Tathagata Das <tathagata.das1565@gmail.com>\nAuthor: wangfei <wangfei1@huawei.com>\n\nCloses #3865 from tdas/print-num and squashes the following commits:\n\ncd34e9e [Tathagata Das] Fix bug\n7c09f16 [Tathagata Das] Merge remote-tracking branch 'apache-github/master' into HEAD\nbb35d1a [Yadong Qi] Update MimaExcludes.scala\nf8098ca [Yadong Qi] Update MimaExcludes.scala\nf6ac3cb [Yadong Qi] Update MimaExcludes.scala\ne4ed897 [Yadong Qi] Update MimaExcludes.scala\n3b9d5cf [wangfei] fix conflicts\nec8a3af [q00251598] move to  Spark 1.3\n26a70c0 [q00251598] extend the Python DStream's print\nb589a4b [q00251598] add another print function","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/bd88b7185358ae60efc83dc6cbb3fb1d2bff6074"},{"sha":"cdccc263b20c1bb27b864411c82cfad7daca1f47","author":{"email":"5662b446b27db1b3677d8356ec5139ec4a65b8eb@darktech.ca","name":"Akhil Das"},"message":"Fixed typos in streaming-kafka-integration.md\n\nChanged projrect to project :)\n\nAuthor: Akhil Das <akhld@darktech.ca>\n\nCloses #3876 from akhld/patch-1 and squashes the following commits:\n\ne0cf9ef [Akhil Das] Fixed typos in streaming-kafka-integration.md","distinct":true,"url":"https://api.github.com/repos/ksakellis/spark/commits/cdccc263b20c1bb27b864411c82cfad7daca1f47"}]}