{"push_id":761421273,"size":524,"distinct_size":524,"ref":"refs/heads/master","head":"c33c0d8d2ba1aee6ca911fcd7951f616e001bfab","before":"08d34e4f3b7a506fa8b0e066685f9e1c0c9b4217","commits":[{"sha":"d4d762f275749a923356cd84de549b14c22cc3eb","author":{"email":"89674bb0dc6f2e03b27640b006f61a9dd0e29d81@hw11853.local","name":"Ram Sriharsha"},"message":"[SPARK-8092] [ML] Allow OneVsRest Classifier feature and label column names to be configurable.\n\nThe base classifier input and output columns are ignored in favor of  the ones specified in OneVsRest.\n\nAuthor: Ram Sriharsha <rsriharsha@hw11853.local>\n\nCloses #6631 from harsha2010/SPARK-8092 and squashes the following commits:\n\n6591dc6 [Ram Sriharsha] add documentation for params\nb7024b1 [Ram Sriharsha] cleanup\nf0e2bfb [Ram Sriharsha] merge with master\n108d3d7 [Ram Sriharsha] merge with master\n4f74126 [Ram Sriharsha] Allow label/ features columns to be configurable","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/d4d762f275749a923356cd84de549b14c22cc3eb"},{"sha":"408e64b284ef8bd6796d815b5eb603312d090b74","author":{"email":"ab56adc4e61f1b489818a700493ea97fb9f9e75e@outlook.com","name":"Wenchen Fan"},"message":"[SPARK-9294][SQL] cleanup comments, code style, naming typo for the new aggregation\n\nfix some comments and code style for https://github.com/apache/spark/pull/7458\n\nAuthor: Wenchen Fan <cloud0fan@outlook.com>\n\nCloses #7619 from cloud-fan/agg-clean and squashes the following commits:\n\n3925457 [Wenchen Fan] one more...\ncc78357 [Wenchen Fan] one more cleanup\n26f6a93 [Wenchen Fan] some minor cleanup for the new aggregation","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/408e64b284ef8bd6796d815b5eb603312d090b74"},{"sha":"cb8c241f05b9ab4ad0cd07df14d454cc5a4554cc","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-9200][SQL] Don't implicitly cast non-atomic types to string type.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #7636 from rxin/complex-string-implicit-cast and squashes the following commits:\n\n3e67327 [Reynold Xin] [SPARK-9200][SQL] Don't implicitly cast non-atomic types to string type.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/cb8c241f05b9ab4ad0cd07df14d454cc5a4554cc"},{"sha":"8fe32b4f7d49607ad5f2479d454b33ab3f079f7c","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[build] Enable memory leak detection for Tungsten.\n\nThis was turned off accidentally in #7591.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #7637 from rxin/enable-mem-leak-detect and squashes the following commits:\n\n34bc3ef [Reynold Xin] Enable memory leak detection for Tungsten.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/8fe32b4f7d49607ad5f2479d454b33ab3f079f7c"},{"sha":"6a7e537f3a4fd5e99a905f9842dc0ad4c348e4fd","author":{"email":"d24118b4297e1ecf0c990d6935f5f9e6e4fd7701@appier.com","name":"Liang-Chi Hsieh"},"message":"[SPARK-8756] [SQL] Keep cached information and avoid re-calculating footers in ParquetRelation2\n\nJIRA: https://issues.apache.org/jira/browse/SPARK-8756\n\nCurrently, in ParquetRelation2, footers are re-read every time refresh() is called. But we can check if it is possibly changed before we do the reading because reading all footers will be expensive when there are too many partitions. This pr fixes this by keeping some cached information to check it.\n\nAuthor: Liang-Chi Hsieh <viirya@appier.com>\n\nCloses #7154 from viirya/cached_footer_parquet_relation and squashes the following commits:\n\n92e9347 [Liang-Chi Hsieh] Fix indentation.\nae0ec64 [Liang-Chi Hsieh] Fix wrong assignment.\nc8fdfb7 [Liang-Chi Hsieh] Fix it.\na52b6d1 [Liang-Chi Hsieh] For comments.\nc2a2420 [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into cached_footer_parquet_relation\nfa5458f [Liang-Chi Hsieh] Use Map to cache FileStatus and do merging previously loaded schema and newly loaded one.\n6ae0911 [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into cached_footer_parquet_relation\n21bbdec [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into cached_footer_parquet_relation\n12a0ed9 [Liang-Chi Hsieh] Add check of FileStatus's modification time.\n186429d [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into cached_footer_parquet_relation\n0ef8caf [Liang-Chi Hsieh] Keep cached information and avoid re-calculating footers.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/6a7e537f3a4fd5e99a905f9842dc0ad4c348e4fd"},{"sha":"6cd28cc21ed585ab8d1e0e7147a1a48b044c9c8e","author":{"email":"5187da0b934cc25eb2201a3ec9206c24b13cb23b@garillot.net","name":"François Garillot"},"message":"[SPARK-9236] [CORE] Make defaultPartitioner not reuse a parent RDD's partitioner if it has 0 partitions\n\nSee also comments on https://issues.apache.org/jira/browse/SPARK-9236\n\nAuthor: François Garillot <francois@garillot.net>\n\nCloses #7616 from huitseeker/issue/SPARK-9236 and squashes the following commits:\n\n217f902 [François Garillot] [SPARK-9236] Make defaultPartitioner not reuse a parent RDD's partitioner if it has 0 partitions","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/6cd28cc21ed585ab8d1e0e7147a1a48b044c9c8e"},{"sha":"dfb18be0366376be3b928dbf4570448c60fe652b","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-9069] [SQL] follow up\n\nAddress comments for #7605\n\ncc rxin\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #7634 from davies/decimal_unlimited2 and squashes the following commits:\n\nb2d8b0d [Davies Liu] add doc and test for DecimalType.isWiderThan\n65b251c [Davies Liu] fix test\n6a91f32 [Davies Liu] fix style\nca9c973 [Davies Liu] address comments","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/dfb18be0366376be3b928dbf4570448c60fe652b"},{"sha":"846cf46282da8f4b87aeee64e407a38cdc80e13b","author":{"email":"5baf40bea4aac2d78f51a81218e1428d32fa1839@intel.com","name":"zhichao.li"},"message":"[SPARK-9238] [SQL] Remove two extra useless entries for bytesOfCodePointInUTF8\n\nOnly a trial thing, not sure if I understand correctly or not but I guess only 2 entries in `bytesOfCodePointInUTF8` for the case of 6 bytes codepoint(1111110x) is enough.\nDetails can be found from https://en.wikipedia.org/wiki/UTF-8 in \"Description\" section.\n\nAuthor: zhichao.li <zhichao.li@intel.com>\n\nCloses #7582 from zhichao-li/utf8 and squashes the following commits:\n\n8bddd01 [zhichao.li] two extra entries","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/846cf46282da8f4b87aeee64e407a38cdc80e13b"},{"sha":"428cde5d1c46adad344255447283dfb9716d65cb","author":{"email":"5187da0b934cc25eb2201a3ec9206c24b13cb23b@garillot.net","name":"François Garillot"},"message":"[SPARK-9250] Make change-scala-version more helpful w.r.t. valid Scala versions\n\nAuthor: François Garillot <francois@garillot.net>\n\nCloses #7595 from huitseeker/issue/SPARK-9250 and squashes the following commits:\n\n80a0218 [François Garillot] [SPARK-9250] Make change-scala-version's usage more explicit, introduce a -h|--help option.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/428cde5d1c46adad344255447283dfb9716d65cb"},{"sha":"3aec9f4e2d8fcce9ddf84ab4d0e10147c18afa16","author":{"email":"bc4a65f227ddcd81bb538c67e810a3e11696e8bf@gmail.com","name":"Yu ISHIKAWA"},"message":"[SPARK-9249] [SPARKR] local variable assigned but may not be used\n\n[[SPARK-9249] local variable assigned but may not be used - ASF JIRA](https://issues.apache.org/jira/browse/SPARK-9249)\n\nhttps://gist.github.com/yu-iskw/0e5b0253c11769457ea5\n\nAuthor: Yu ISHIKAWA <yuu.ishikawa@gmail.com>\n\nCloses #7640 from yu-iskw/SPARK-9249 and squashes the following commits:\n\n7a51cab [Yu ISHIKAWA] [SPARK-9249][SparkR] local variable assigned but may not be used","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/3aec9f4e2d8fcce9ddf84ab4d0e10147c18afa16"},{"sha":"431ca39be51352dfcdacc87de7e64c2af313558d","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-9285][SQL] Remove InternalRow's inheritance from Row.\n\nI also changed InternalRow's size/length function to numFields, to make it more obvious that it is not about bytes, but the number of fields.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #7626 from rxin/internalRow and squashes the following commits:\n\ne124daf [Reynold Xin] Fixed test case.\n805ceb7 [Reynold Xin] Commented out the failed test suite.\nf8a9ca5 [Reynold Xin] Fixed more bugs. Still at least one more remaining.\n76d9081 [Reynold Xin] Fixed data sources.\n7807f70 [Reynold Xin] Fixed DataFrameSuite.\ncb60cd2 [Reynold Xin] Code review & small bug fixes.\n0a2948b [Reynold Xin] Fixed style.\n3280d03 [Reynold Xin] [SPARK-9285][SQL] Remove InternalRow's inheritance from Row.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/431ca39be51352dfcdacc87de7e64c2af313558d"},{"sha":"c8d71a4183dfc83ff257047857af0b6d66c6b90d","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-9305] Rename org.apache.spark.Row to Item.\n\nIt's a thing used in test cases, but named Row. Pretty annoying because everytime I search for Row, it shows up before the Spark SQL Row, which is what a developer wants most of the time.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #7638 from rxin/remove-row and squashes the following commits:\n\naeda52d [Reynold Xin] [SPARK-9305] Rename org.apache.spark.Row to Item.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/c8d71a4183dfc83ff257047857af0b6d66c6b90d"},{"sha":"c2b50d693e469558e3b3c9cbb9d76089d259b587","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-9292] Analysis should check that join conditions' data types are BooleanType\n\nThis patch adds an analysis check to ensure that join conditions' data types are BooleanType. This check is necessary in order to report proper errors for non-boolean DataFrame join conditions.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #7630 from JoshRosen/SPARK-9292 and squashes the following commits:\n\naec6c7b [Josh Rosen] Check condition type in resolved()\n75a3ea6 [Josh Rosen] Fix SPARK-9292.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/c2b50d693e469558e3b3c9cbb9d76089d259b587"},{"sha":"e25312451322969ad716dddf8248b8c17f68323b","author":{"email":"8757c4a255384de9fc2175f17a84f948e7081d30@gmail.com","name":"MechCoder"},"message":"[SPARK-9222] [MLlib] Make class instantiation variables in DistributedLDAModel private[clustering]\n\nThis makes it easier to test all the class variables of the DistributedLDAmodel.\n\nAuthor: MechCoder <manojkumarsivaraj334@gmail.com>\n\nCloses #7573 from MechCoder/lda_test and squashes the following commits:\n\n2f1a293 [MechCoder] [SPARK-9222] [MLlib] Make class instantiation variables in DistributedLDAModel private[clustering]","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/e25312451322969ad716dddf8248b8c17f68323b"},{"sha":"6aceaf3d62ee335570ddc07ccaf07e8c3776f517","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-9295] Analysis should detect sorting on unsupported column types\n\nThis patch extends CheckAnalysis to throw errors for queries that try to sort on unsupported column types, such as ArrayType.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #7633 from JoshRosen/SPARK-9295 and squashes the following commits:\n\n23b2fbf [Josh Rosen] Embed function in foreach\nbfe1451 [Josh Rosen] Update to allow sorting by null literals\n2f1b802 [Josh Rosen] Add analysis rule to detect sorting on unsupported column types (SPARK-9295)","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/6aceaf3d62ee335570ddc07ccaf07e8c3776f517"},{"sha":"8399ba14873854ab2f80a0ccaf6adba499060365","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-9261] [STREAMING] Avoid calling APIs that expose shaded classes.\n\nDoing this may cause weird errors when tests are run on maven, depending\non the flags used. Instead, expose the needed functionality through methods\nthat do not expose shaded classes.\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #7601 from vanzin/SPARK-9261 and squashes the following commits:\n\n4f64a16 [Marcelo Vanzin] [SPARK-9261] [streaming] Avoid calling APIs that expose shaded classes.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/8399ba14873854ab2f80a0ccaf6adba499060365"},{"sha":"9a11396113d4bb0e76e0520df4fc58e7a8ec9f69","author":{"email":"9d01406527e5c15536e8dcb4e0c5005dca46786b@netflix.com","name":"Cheolsoo Park"},"message":"[SPARK-9270] [PYSPARK] allow --name option in pyspark\n\nThis is continuation of #7512 which added `--name` option to spark-shell. This PR adds the same option to pyspark.\n\nNote that `--conf spark.app.name` in command-line has no effect in spark-shell and pyspark. Instead, `--name` must be used. This is in fact inconsistency with spark-sql which doesn't accept `--name` option while it accepts `--conf spark.app.name`. I am not fixing this inconsistency in this PR. IMO, one of `--name` and `--conf spark.app.name` is needed not both. But since I cannot decide which to choose, I am not making any change here.\n\nAuthor: Cheolsoo Park <cheolsoop@netflix.com>\n\nCloses #7610 from piaozhexiu/SPARK-9270 and squashes the following commits:\n\n763e86d [Cheolsoo Park] Update windows script\n400b7f9 [Cheolsoo Park] Allow --name option to pyspark","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/9a11396113d4bb0e76e0520df4fc58e7a8ec9f69"},{"sha":"64135cbb3363e3b74dad3c0498cb9959c047d381","author":{"email":"d24118b4297e1ecf0c990d6935f5f9e6e4fd7701@appier.com","name":"Liang-Chi Hsieh"},"message":"[SPARK-9067] [SQL] Close reader in NewHadoopRDD early if there is no more data\n\nJIRA: https://issues.apache.org/jira/browse/SPARK-9067\n\nAccording to the description of the JIRA ticket, calling `reader.close()` only after the task is finished will cause memory and file open limit problem since these resources are occupied even we don't need that anymore.\n\nThis PR simply closes the reader early when we know there is no more data to read.\n\nAuthor: Liang-Chi Hsieh <viirya@appier.com>\n\nCloses #7424 from viirya/close_reader and squashes the following commits:\n\n3ff64e5 [Liang-Chi Hsieh] For comments.\n3d20267 [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into close_reader\ne152182 [Liang-Chi Hsieh] For comments.\n5116cbe [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into close_reader\n3ceb755 [Liang-Chi Hsieh] For comments.\ne34d98e [Liang-Chi Hsieh] For comments.\n50ed729 [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into close_reader\n216912f [Liang-Chi Hsieh] Fix it.\nf429016 [Liang-Chi Hsieh] Release reader if we don't need it.\na305621 [Liang-Chi Hsieh] Merge remote-tracking branch 'upstream/master' into close_reader\n67569da [Liang-Chi Hsieh] Close reader early if there is no more data.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/64135cbb3363e3b74dad3c0498cb9959c047d381"},{"sha":"a400ab516fa93185aa683a596f9d7c6c1a02f330","author":{"email":"8757c4a255384de9fc2175f17a84f948e7081d30@gmail.com","name":"MechCoder"},"message":"[SPARK-7045] [MLLIB] Avoid intermediate representation when creating model\n\nWord2Vec used to convert from an Array[Float] representation to a Map[String, Array[Float]] and then back to an Array[Float] through Word2VecModel.\n\nThis prevents this conversion while still supporting the older method of supplying a Map.\n\nAuthor: MechCoder <manojkumarsivaraj334@gmail.com>\n\nCloses #5748 from MechCoder/spark-7045 and squashes the following commits:\n\ne308913 [MechCoder] move docs\n5703116 [MechCoder] minor\nfa04313 [MechCoder] style fixes\nb1d61c4 [MechCoder] better errors and tests\n3b32c8c [MechCoder] [SPARK-7045] Avoid intermediate representation when creating model","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/a400ab516fa93185aa683a596f9d7c6c1a02f330"},{"sha":"f99cb5615cbc0b469d52af6bd08f8bf888af58f3","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-9330][SQL] Create specialized getStruct getter in InternalRow.\n\nAlso took the chance to rearrange some of the methods in UnsafeRow to group static/private/public things together.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #7654 from rxin/getStruct and squashes the following commits:\n\nb491a09 [Reynold Xin] Fixed typo.\n48d77e5 [Reynold Xin] [SPARK-9330][SQL] Create specialized getStruct getter in InternalRow.","distinct":true,"url":"https://api.github.com/repos/rogerdai16/spark/commits/f99cb5615cbc0b469d52af6bd08f8bf888af58f3"}]}