{"push_id":726190084,"size":10,"distinct_size":10,"ref":"refs/heads/SPARK-9054","head":"8060594c6f83846850484465216828cbf5720952","before":"5eb7222a2a8e4654064bc43facdadefc7e527281","commits":[{"sha":"4b5cfc988f23988c2334882a255d494fc93d252e","author":{"email":"d24118b4297e1ecf0c990d6935f5f9e6e4fd7701@gmail.com","name":"Liang-Chi Hsieh"},"message":"[SPARK-8800] [SQL] Fix inaccurate precision/scale of Decimal division operation\n\nJIRA: https://issues.apache.org/jira/browse/SPARK-8800\n\nPreviously, we turn to Java BigDecimal's divide with specified ROUNDING_MODE to avoid non-terminating decimal expansion problem. However, as JihongMA reported, for the division operation on some specific values, we get inaccurate results.\n\nAuthor: Liang-Chi Hsieh <viirya@gmail.com>\n\nCloses #7212 from viirya/fix_decimal4 and squashes the following commits:\n\n4205a0a [Liang-Chi Hsieh] Fix inaccuracy precision/scale of Decimal division operation.","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/4b5cfc988f23988c2334882a255d494fc93d252e"},{"sha":"740b034f1ca885a386f5a9ef7e0c81c714b047ff","author":{"email":"291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com","name":"Sean Owen"},"message":"[SPARK-4362] [MLLIB] Make prediction probability available in NaiveBayesModel\n\nAdd predictProbabilities to Naive Bayes, return class probabilities.\n\nContinues https://github.com/apache/spark/pull/6761\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #7376 from srowen/SPARK-4362 and squashes the following commits:\n\n23d5a76 [Sean Owen] Fix model.labels -> model.theta\n95d91fb [Sean Owen] Check that predicted probabilities sum to 1\nb32d1c8 [Sean Owen] Add predictProbabilities to Naive Bayes, return class probabilities","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/740b034f1ca885a386f5a9ef7e0c81c714b047ff"},{"sha":"11e5c372862ec00e57460b37ccfee51c6d93c5f7","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-8962] Add Scalastyle rule to ban direct use of Class.forName; fix existing uses\n\nThis pull request adds a Scalastyle regex rule which fails the style check if `Class.forName` is used directly.  `Class.forName` always loads classes from the default / system classloader, but in a majority of cases, we should be using Spark's own `Utils.classForName` instead, which tries to load classes from the current thread's context classloader and falls back to the classloader which loaded Spark when the context classloader is not defined.\n\n<!-- Reviewable:start -->\n[<img src=\"https://reviewable.io/review_button.png\" height=40 alt=\"Review on Reviewable\"/>](https://reviewable.io/reviews/apache/spark/7350)\n<!-- Reviewable:end -->\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #7350 from JoshRosen/ban-Class.forName and squashes the following commits:\n\ne3e96f7 [Josh Rosen] Merge remote-tracking branch 'origin/master' into ban-Class.forName\nc0b7885 [Josh Rosen] Hopefully fix the last two cases\nd707ba7 [Josh Rosen] Fix uses of Class.forName that I missed in my first cleanup pass\n046470d [Josh Rosen] Merge remote-tracking branch 'origin/master' into ban-Class.forName\n62882ee [Josh Rosen] Fix uses of Class.forName or add exclusion.\nd9abade [Josh Rosen] Add stylechecker rule to ban uses of Class.forName","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/11e5c372862ec00e57460b37ccfee51c6d93c5f7"},{"sha":"e965a798d09a9fba61b104c5cc0b65cdc28d27f6","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-9045] Fix Scala 2.11 build break in UnsafeExternalRowSorter\n\nThis fixes a compilation break in under Scala 2.11:\n\n```\n[error] /home/jenkins/workspace/Spark-Master-Scala211-Compile/sql/catalyst/src/main/java/org/apache/spark/sql/execution/UnsafeExternalRowSorter.java:135: error: <anonymous org.apache.spark.sql.execution.UnsafeExternalRowSorter$1> is not abstract and does not override abstract method <B>minBy(Function1<InternalRow,B>,Ordering<B>) in TraversableOnce\n[error]       return new AbstractScalaRowIterator() {\n[error]                                             ^\n[error]   where B,A are type-variables:\n[error]     B extends Object declared in method <B>minBy(Function1<A,B>,Ordering<B>)\n[error]     A extends Object declared in interface TraversableOnce\n[error] 1 error\n```\n\nThe workaround for this is to make `AbstractScalaRowIterator` into a concrete class.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #7405 from JoshRosen/SPARK-9045 and squashes the following commits:\n\ncbcbb4c [Josh Rosen] Forgot that we can't use the ??? operator anymore\n577ba60 [Josh Rosen] [SPARK-9045] Fix Scala 2.11 build break in UnsafeExternalRowSorter.","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/e965a798d09a9fba61b104c5cc0b65cdc28d27f6"},{"sha":"cc57d705e732aefc2f3d3f438e84d71705b2eb65","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-9050] [SQL] Remove unused newOrdering argument from Exchange (cleanup after SPARK-8317)\n\nSPARK-8317 changed the SQL Exchange operator so that it no longer pushed sorting into Spark's shuffle layer, a change which allowed more efficient SQL-specific sorters to be used.\n\nThis patch performs some leftover cleanup based on those changes:\n\n- Exchange's constructor should no longer accept a `newOrdering` since it's no longer used and no longer works as expected.\n- `addOperatorsIfNecessary` looked at shuffle input's output ordering to decide whether to sort, but this is the wrong node to be examining: it needs to look at whether the post-shuffle node has the right ordering, since shuffling will not preserve row orderings.  Thanks to davies for spotting this.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #7407 from JoshRosen/SPARK-9050 and squashes the following commits:\n\ne70be50 [Josh Rosen] No need to wrap line\ne866494 [Josh Rosen] Refactor addOperatorsIfNecessary to make code clearer\n2e467da [Josh Rosen] Remove `newOrdering` from Exchange.","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/cc57d705e732aefc2f3d3f438e84d71705b2eb65"},{"sha":"f957796c4b3c3cd95edfc64500a045f7e810ee87","author":{"email":"2c0006ca50109642a00c809a04a8dbcd53363bf2@gmail.com","name":"huangzhaowei"},"message":"[SPARK-8820] [STREAMING] Add a configuration to set checkpoint dir.\n\nAdd a configuration to set checkpoint directory  for convenience to user.\n[Jira Address](https://issues.apache.org/jira/browse/SPARK-8820)\n\nAuthor: huangzhaowei <carlmartinmax@gmail.com>\n\nCloses #7218 from SaintBacchus/SPARK-8820 and squashes the following commits:\n\nd49fe4b [huangzhaowei] Rename the configuration name\n66ea47c [huangzhaowei] Add the unit test.\ndd0acc1 [huangzhaowei] [SPARK-8820][Streaming] Add a configuration to set checkpoint dir.","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/f957796c4b3c3cd95edfc64500a045f7e810ee87"},{"sha":"bb870e72f42b6ce8d056df259f6fcf41808d7ed2","author":{"email":"79478dbbc8641d1d6545f348f655464aefbda99b@intel.com","name":"jerryshao"},"message":"[SPARK-5523] [CORE] [STREAMING] Add a cache for hostname in TaskMetrics to decrease the memory usage and GC overhead\n\nHostname in TaskMetrics will be created through deserialization, mostly the number of hostname is only the order of number of cluster node, so adding a cache layer to dedup the object could reduce the memory usage and alleviate GC overhead, especially for long-running and fast job generation applications like Spark Streaming.\n\nAuthor: jerryshao <saisai.shao@intel.com>\nAuthor: Saisai Shao <saisai.shao@intel.com>\n\nCloses #5064 from jerryshao/SPARK-5523 and squashes the following commits:\n\n3e2412a [jerryshao] Address the comments\nb092a81 [Saisai Shao] Add a pool to cache the hostname","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/bb870e72f42b6ce8d056df259f6fcf41808d7ed2"},{"sha":"845dd19e655d93feaec72b66bdd5e460f89e75c3","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"Merge remote-tracking branch 'origin/master' into SPARK-9054","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/845dd19e655d93feaec72b66bdd5e460f89e75c3"},{"sha":"9e2abbb0bd0d2896139fc90d74b05e296ea91008","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"Add wrapper to allow RangePartitioner's Ordering to be serialized","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/9e2abbb0bd0d2896139fc90d74b05e296ea91008"},{"sha":"8060594c6f83846850484465216828cbf5720952","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"Merge remote-tracking branch 'josh/SPARK-9054' into SPARK-9054","distinct":true,"url":"https://api.github.com/repos/JoshRosen/spark/commits/8060594c6f83846850484465216828cbf5720952"}]}