{"push_id":776410096,"size":475,"distinct_size":475,"ref":"refs/heads/master","head":"b5cbd9a519179412682bf22e29e297979afcd1a1","before":"26ff2e0c365e3ebe2e300bf98a417d5486d40fff","commits":[{"sha":"6a47114bc297f0bce874e425feb1c24a5c26cef0","author":{"email":"04ade2e3a4ce65c5ee0cbfa08dacc69e7d690176@apache.org","name":"navis.ryu"},"message":"[SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first\n\n    case cs  CombineSum(expr) =>\n        val calcType = expr.dataType\n          expr.dataType match {\n            case DecimalType.Fixed(_, _) =>\n              DecimalType.Unlimited\n            case _ =>\n              expr.dataType\n          }\ncalcType is always expr.dataType. credits are all belong to IntelliJ\n\nAuthor: navis.ryu <navis@apache.org>\n\nCloses #6736 from navis/SPARK-8285 and squashes the following commits:\n\n20382c1 [navis.ryu] [SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/6a47114bc297f0bce874e425feb1c24a5c26cef0"},{"sha":"4e42842e82e058d54329bd66185d8a7e77ab335a","author":{"email":"ab56adc4e61f1b489818a700493ea97fb9f9e75e@outlook.com","name":"Wenchen Fan"},"message":"[SPARK-8164] transformExpressions should support nested expression sequence\n\nCurrently we only support `Seq[Expression]`, we should handle cases like `Seq[Seq[Expression]]` so that we can remove the unnecessary `GroupExpression`.\n\nAuthor: Wenchen Fan <cloud0fan@outlook.com>\n\nCloses #6706 from cloud-fan/clean and squashes the following commits:\n\n60a1193 [Wenchen Fan] support nested expression sequence and remove GroupExpression","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/4e42842e82e058d54329bd66185d8a7e77ab335a"},{"sha":"9fe3adccef687c92ff1ac17d946af089c8e28d66","author":{"email":"50c4b4c4c26f83ee4bbc45a85db1c3c80b5aecdb@intel.com","name":"Cheng Hao"},"message":"[SPARK-8248][SQL] string function: length\n\nAuthor: Cheng Hao <hao.cheng@intel.com>\n\nCloses #6724 from chenghao-intel/length and squashes the following commits:\n\naaa3c31 [Cheng Hao] revert the additional change\n97148a9 [Cheng Hao] remove the codegen testing temporally\nae08003 [Cheng Hao] update the comments\n1eb1fd1 [Cheng Hao] simplify the code as commented\n3e92d32 [Cheng Hao] use the selectExpr in unit test intead of SQLQuery\n3c729aa [Cheng Hao] fix bug for constant null value in codegen\n3641f06 [Cheng Hao] keep the length() method for registered function\n8e30171 [Cheng Hao] update the code as comment\ndb604ae [Cheng Hao] Add code gen support\n548d2ef [Cheng Hao] register the length()\n09a0738 [Cheng Hao] add length support","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/9fe3adccef687c92ff1ac17d946af089c8e28d66"},{"sha":"2758ff0a96f03a61e10999b2462acf7a13236b7c","author":{"email":"c2bf8a29ab4e9ca2b842ca45622f95cb9164f566@intel.com","name":"Daoyuan Wang"},"message":"[SPARK-8217] [SQL] math function log2\n\nAuthor: Daoyuan Wang <daoyuan.wang@intel.com>\n\nThis patch had conflicts when merged, resolved by\nCommitter: Reynold Xin <rxin@databricks.com>\n\nCloses #6718 from adrian-wang/udflog2 and squashes the following commits:\n\n3909f48 [Daoyuan Wang] math function: log2","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/2758ff0a96f03a61e10999b2462acf7a13236b7c"},{"sha":"a777eb04bf981312b640326607158f78dd4163cd","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"[HOTFIX] Adding more contributor name bindings","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/a777eb04bf981312b640326607158f78dd4163cd"},{"sha":"e84545fa771dde90de5675a9c551fe287af6f7fb","author":{"email":"cbb7353e6d953ef360baf960c122346276c6e320@databricks.com","name":"Patrick Wendell"},"message":"[HOTFIX] Fixing errors in name mappings","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/e84545fa771dde90de5675a9c551fe287af6f7fb"},{"sha":"6b68366df345d4572cf138f9efe17e23d0d1971e","author":{"email":"130a8c2510da53d4596e307e12003003207f1b66@uk.ibm.com","name":"Adam Roberts"},"message":"[SPARK-8289] Specify stack size for consistency with Java tests - resolves test failures\n\nThis change is a simple one and specifies a stack size of 4096k instead of the vendor default for Java tests (the defaults vary between Java vendors). This remedies test failures observed with JavaALSSuite with IBM and Oracle Java owing to a lower default size in comparison to the size with OpenJDK. 4096k is a suitable default where the tests pass with each Java vendor tested. The alternative is to reduce the number of iterations in the test (no observed failures with 5 iterations instead of 15).\n\n-Xss works with Oracle's HotSpot VM, IBM's J9 VM and OpenJDK (IcedTea).\n\nI have ensured this does not have any negative implications for other tests.\n\nAuthor: Adam Roberts <aroberts@uk.ibm.com>\nAuthor: a-roberts <aroberts@uk.ibm.com>\n\nCloses #6727 from a-roberts/IncJavaStackSize and squashes the following commits:\n\nab40aea [Adam Roberts] Specify stack size for SBT builds\n5032d8d [a-roberts] Update pom.xml","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/6b68366df345d4572cf138f9efe17e23d0d1971e"},{"sha":"424b0075a1a31c251451c6a75c6ba8e81c39453d","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-6411] [SQL] [PySpark] support date/datetime with timezone in Python\n\nSpark SQL does not support timezone, and Pyrolite does not support timezone well. This patch will convert datetime into POSIX timestamp (without confusing of timezone), which is used by SQL. If the datetime object does not have timezone, it's treated as local time.\n\nThe timezone in RDD will be lost after one round trip, all the datetime from SQL will be local time.\n\nBecause of Pyrolite, datetime from SQL only has precision as 1 millisecond.\n\nThis PR also drop the timezone in date, convert it to number of days since epoch (used in SQL).\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #6250 from davies/tzone and squashes the following commits:\n\n44d8497 [Davies Liu] add timezone support for DateType\n99d9d9c [Davies Liu] use int for timestamp\n10aa7ca [Davies Liu] Merge branch 'master' of github.com:apache/spark into tzone\n6a29aa4 [Davies Liu] support datetime with timezone","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/424b0075a1a31c251451c6a75c6ba8e81c39453d"},{"sha":"1191c3efc605d9c6d1df4b38ddae8d210a361b5b","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-8305] [SPARK-8190] [SQL] improve codegen\n\nThis PR fix a few small issues about codgen:\n\n1. cast decimal to boolean\n2. do not inline literal with null\n3. improve SpecificRow.equals()\n4. test expressions with optimized express\n5. fix compare with BinaryType\n\ncc rxin chenghao-intel\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #6755 from davies/fix_codegen and squashes the following commits:\n\nef27343 [Davies Liu] address comments\n6617ea6 [Davies Liu] fix scala tyle\n70b7dda [Davies Liu] improve codegen","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/1191c3efc605d9c6d1df4b38ddae8d210a361b5b"},{"sha":"c8d551d546979e126c91925487e30c353185e3ba","author":{"email":"ad63b47dfac1951bde2b661ba9fbbe123f686bee@cs.berkeley.edu","name":"Shivaram Venkataraman"},"message":"[SPARK-8310] [EC2] Updates the master branch EC2 versions\n\nWill send another PR for `branch-1.4`\n\nAuthor: Shivaram Venkataraman <shivaram@cs.berkeley.edu>\n\nCloses #6764 from shivaram/SPARK-8310 and squashes the following commits:\n\nd8cd3b3 [Shivaram Venkataraman] This updates the master branch EC2 versions","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/c8d551d546979e126c91925487e30c353185e3ba"},{"sha":"040f223c5b9ca724c9f2b4abb59c21b3a23720ba","author":{"email":"50c4b4c4c26f83ee4bbc45a85db1c3c80b5aecdb@intel.com","name":"Cheng Hao"},"message":"[SPARK-7915] [SQL] Support specifying the column list for target table in CTAS\n\n```\ncreate table t1 (a int, b string) as select key, value from src;\n\ndesc t1;\nkey\tint\tNULL\nvalue\tstring\tNULL\n```\n\nThus Hive doesn't support specifying the column list for target table in CTAS, however, we should either throwing exception explicity, or supporting the this feature, we just pick up the later one, which seems useful and straightforward.\n\nAuthor: Cheng Hao <hao.cheng@intel.com>\n\nCloses #6458 from chenghao-intel/ctas_column and squashes the following commits:\n\nd1fa9b6 [Cheng Hao] bug in unittest\n4e701aa [Cheng Hao] update as feedback\nf305ec1 [Cheng Hao] support specifying the column list for target table in CTAS","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/040f223c5b9ca724c9f2b4abb59c21b3a23720ba"},{"sha":"95690a17d328f205c3398b9b477b4072b6fe908f","author":{"email":"5d14e010a5b345b784fd604efcc6cebb79f97468@gmail.com","name":"zsxwing"},"message":"[SPARK-7444] [TESTS] Eliminate noisy css warn/error logs for UISeleniumSuite\n\nEliminate the following noisy logs for `UISeleniumSuite`:\n```\n15/05/07 10:09:50.196 pool-1-thread-1-ScalaTest-running-UISeleniumSuite WARN DefaultCssErrorHandler: CSS error: 'http://192.168.0.170:4040/static/bootstrap.min.css' [793:167] Error in style rule. (Invalid token \"*\". Was expecting one of: <EOF>, <S>, <IDENT>, \"}\", \";\".)\n15/05/07 10:09:50.196 pool-1-thread-1-ScalaTest-running-UISeleniumSuite WARN DefaultCssErrorHandler: CSS warning: 'http://192.168.0.170:4040/static/bootstrap.min.css' [793:167] Ignoring the following declarations in this rule.\n15/05/07 10:09:50.197 pool-1-thread-1-ScalaTest-running-UISeleniumSuite WARN DefaultCssErrorHandler: CSS error: 'http://192.168.0.170:4040/static/bootstrap.min.css' [799:325] Error in style rule. (Invalid token \"*\". Was expecting one of: <EOF>, <S>, <IDENT>, \"}\", \";\".)\n15/05/07 10:09:50.197 pool-1-thread-1-ScalaTest-running-UISeleniumSuite WARN DefaultCssErrorHandler: CSS warning: 'http://192.168.0.170:4040/static/bootstrap.min.css' [799:325] Ignoring the following declarations in this rule.\n15/05/07 10:09:50.198 pool-1-thread-1-ScalaTest-running-UISeleniumSuite WARN DefaultCssErrorHandler: CSS error: 'http://192.168.0.170:4040/static/bootstrap.min.css' [805:18] Error in style rule. (Invalid token \"*\". Was expecting one of: <EOF>, <S>, <IDENT>, \"}\", \";\".)\n15/05/07 10:09:50.198 pool-1-thread-1-ScalaTest-running-UISeleniumSuite WARN DefaultCssErrorHandler: CSS warning: 'http://192.168.0.170:4040/static/bootstrap.min.css' [805:18] Ignoring the following declarations in this rule.\n```\n\nAuthor: zsxwing <zsxwing@gmail.com>\n\nCloses #5983 from zsxwing/SPARK-7444 and squashes the following commits:\n\n4202728 [zsxwing] Add SparkUICssErrorHandler for all tests\nd1398ad [zsxwing] Merge remote-tracking branch 'origin/master' into SPARK-7444\n7bb7f11 [zsxwing] Merge branch 'master' into SPARK-7444\na59f40e [zsxwing] Eliminate noisy css warn/error logs for UISeleniumSuite","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/95690a17d328f205c3398b9b477b4072b6fe908f"},{"sha":"9cbdf31ec1399d4d43a1863c15688ce78b6dfd92","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-6511] [docs] Fix example command in hadoop-provided docs.\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #6766 from vanzin/SPARK-6511 and squashes the following commits:\n\n49f0f67 [Marcelo Vanzin] [SPARK-6511] [docs] Fix example command in hadoop-provided docs.","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/9cbdf31ec1399d4d43a1863c15688ce78b6dfd92"},{"sha":"7d669a56ffc7a4f5827830ef3c27d45cc0e8774f","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-8286] Rewrite UTF8String in Java and move it into unsafe package.\n\nUnit test is still in Scala.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #6738 from rxin/utf8string-java and squashes the following commits:\n\n562dc6e [Reynold Xin] Flag...\n98e600b [Reynold Xin] Another try with encoding setting ..\ncfa6bdf [Reynold Xin] Merge branch 'master' into utf8string-java\na3b124d [Reynold Xin] Try different UTF-8 encoded characters.\n1ff7c82 [Reynold Xin] Enable UTF-8 encoding.\n82d58cc [Reynold Xin] Reset run-tests.\n2cb3c69 [Reynold Xin] Use utf-8 encoding in set bytes.\n53f8ef4 [Reynold Xin] Hack Jenkins to run one test.\n9a48e8d [Reynold Xin] Fixed runtime compilation error.\n911c450 [Reynold Xin] Moved unit test also to Java.\n4eff7bd [Reynold Xin] Improved unit test coverage.\n8e89a3c [Reynold Xin] Fixed tests.\n77c64bd [Reynold Xin] Fixed string type codegen.\nffedb62 [Reynold Xin] Code review feedback.\n0967ce6 [Reynold Xin] Fixed import ordering.\n45a123d [Reynold Xin] [SPARK-8286] Rewrite UTF8String in Java and move it into unsafe package.","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/7d669a56ffc7a4f5827830ef3c27d45cc0e8774f"},{"sha":"7914c720bf7447e8c9d96d564eafd6b687d2fc1a","author":{"email":"7acf0e49bc9acc50da7e38d41884e81d5f2df305@qq.com","name":"Zhongshuai Pei"},"message":"[SPARK-7824] [SQL] Collapse operator reordering and constant folding into a single batch.\n\nSQL\n```\nselect * from tableA join tableB on (a > 3 and b = d) or (a > 3 and b = e)\n```\nPlan before modify\n```\n== Optimized Logical Plan ==\nProject [a#293,b#294,c#295,d#296,e#297]\n Join Inner, Some(((a#293 > 3) && ((b#294 = d#296) || (b#294 = e#297))))\n  MetastoreRelation default, tablea, None\n  MetastoreRelation default, tableb, None\n```\nPlan after modify\n```\n== Optimized Logical Plan ==\nProject [a#293,b#294,c#295,d#296,e#297]\n Join Inner, Some(((b#294 = d#296) || (b#294 = e#297)))\n  Filter (a#293 > 3)\n   MetastoreRelation default, tablea, None\n  MetastoreRelation default, tableb, None\n```\n\nCombineLimits ==> Limit(If(LessThan(ne, le), ne, le), grandChild) and LessThan is in BooleanSimplification ,  so CombineLimits  must before BooleanSimplification and BooleanSimplification must before PushPredicateThroughJoin.\n\nAuthor: Zhongshuai Pei <799203320@qq.com>\nAuthor: DoingDone9 <799203320@qq.com>\n\nCloses #6351 from DoingDone9/master and squashes the following commits:\n\n20de7be [Zhongshuai Pei] Update Optimizer.scala\n7bc7d28 [Zhongshuai Pei] Merge pull request #17 from apache/master\n0ba5f42 [Zhongshuai Pei] Update Optimizer.scala\nf8b9314 [Zhongshuai Pei] Update FilterPushdownSuite.scala\nc529d9f [Zhongshuai Pei] Update FilterPushdownSuite.scala\nae3af6d [Zhongshuai Pei] Update FilterPushdownSuite.scala\na04ffae [Zhongshuai Pei] Update Optimizer.scala\n11beb61 [Zhongshuai Pei] Update FilterPushdownSuite.scala\nf2ee5fe [Zhongshuai Pei] Update Optimizer.scala\nbe6b1d5 [Zhongshuai Pei] Update Optimizer.scala\nb01e622 [Zhongshuai Pei] Merge pull request #15 from apache/master\n8df716a [Zhongshuai Pei] Update FilterPushdownSuite.scala\nd98bc35 [Zhongshuai Pei] Update FilterPushdownSuite.scala\nfa65718 [Zhongshuai Pei] Update Optimizer.scala\nab8e9a6 [Zhongshuai Pei] Merge pull request #14 from apache/master\n14952e2 [Zhongshuai Pei] Merge pull request #13 from apache/master\nf03fe7f [Zhongshuai Pei] Merge pull request #12 from apache/master\nf12fa50 [Zhongshuai Pei] Merge pull request #10 from apache/master\nf61210c [Zhongshuai Pei] Merge pull request #9 from apache/master\n34b1a9a [Zhongshuai Pei] Merge pull request #8 from apache/master\n802261c [DoingDone9] Merge pull request #7 from apache/master\nd00303b [DoingDone9] Merge pull request #6 from apache/master\n98b134f [DoingDone9] Merge pull request #5 from apache/master\n161cae3 [DoingDone9] Merge pull request #4 from apache/master\nc87e8b6 [DoingDone9] Merge pull request #3 from apache/master\ncb1852d [DoingDone9] Merge pull request #2 from apache/master\nc3f046f [DoingDone9] Merge pull request #1 from apache/master","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/7914c720bf7447e8c9d96d564eafd6b687d2fc1a"},{"sha":"337c16d57e40cb4967bf85269baae14745f161db","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SQL] Miscellaneous SQL/DF expression changes.\n\nSPARK-8201 conditional function: if\nSPARK-8205 conditional function: nvl\nSPARK-8208 math function: ceiling\nSPARK-8210 math function: degrees\nSPARK-8211 math function: radians\nSPARK-8219 math function: negative\nSPARK-8216 math function: rename log -> ln\nSPARK-8222 math function: alias power / pow\nSPARK-8225 math function: alias sign / signum\nSPARK-8228 conditional function: isnull\nSPARK-8229 conditional function: isnotnull\nSPARK-8250 string function: alias lower/lcase\nSPARK-8251 string function: alias upper / ucase\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #6754 from rxin/expressions-misc and squashes the following commits:\n\n35fce15 [Reynold Xin] Removed println.\n2647067 [Reynold Xin] Promote to string type.\n3c32bbc [Reynold Xin] Fixed if.\nde827ac [Reynold Xin] Fixed style\nb201cd4 [Reynold Xin] Removed if.\n6b21a9b [Reynold Xin] [SQL] Miscellaneous SQL/DF expression changes.","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/337c16d57e40cb4967bf85269baae14745f161db"},{"sha":"767cc94ca6d397ba19226996ccb3c8e57083c549","author":{"email":"50c4b4c4c26f83ee4bbc45a85db1c3c80b5aecdb@intel.com","name":"Cheng Hao"},"message":"[SPARK-7158] [SQL] Fix bug of cached data cannot be used in collect() after cache()\n\nWhen df.cache() method called, the `withCachedData` of `QueryExecution` has been created, which mean it will not look up the cached tables when action method called afterward.\n\nAuthor: Cheng Hao <hao.cheng@intel.com>\n\nCloses #5714 from chenghao-intel/SPARK-7158 and squashes the following commits:\n\n58ea8aa [Cheng Hao] style issue\n2bf740f [Cheng Hao] create new QueryExecution instance for CacheManager\na5647d9 [Cheng Hao] hide the queryExecution of DataFrame\nfbfd3c5 [Cheng Hao] make the DataFrame.queryExecution mutable for cache/persist/unpersist","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/767cc94ca6d397ba19226996ccb3c8e57083c549"},{"sha":"b9d177c5110cd054fdb9bcbeeb5f4ca9aa645dc1","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-8317] [SQL] Do not push sort into shuffle in Exchange operator\n\nIn some cases, Spark SQL pushes sorting operations into the shuffle layer by specifying a key ordering as part of the shuffle dependency. I think that we should not do this:\n\n- Since we do not delegate aggregation to Spark's shuffle, specifying the keyOrdering as part of the shuffle has no effect on the shuffle map side.\n- By performing the shuffle ourselves (by inserting a sort operator after the shuffle instead), we can use the Exchange planner to choose specialized sorting implementations based on the types of rows being sorted.\n- We can remove some complexity from SqlSerializer2 by not requiring it to know about sort orderings, since SQL's own sort operators will already perform the necessary defensive copying.\n\nThis patch removes Exchange's `canSortWithShuffle` path and the associated code in `SqlSerializer2`.  Shuffles that used to go through the `canSortWithShuffle` path would always wind up using Spark's `ExternalSorter` (inside of `HashShuffleReader`); to avoid a performance regression as a result of handling these shuffles ourselves, I've changed the SQLConf defaults so that external sorting is enabled by default.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #6772 from JoshRosen/SPARK-8317 and squashes the following commits:\n\nebf9c0f [Josh Rosen] Do not push sort into shuffle in Exchange operator\nbf3b4c8 [Josh Rosen] Enable external sort by default","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/b9d177c5110cd054fdb9bcbeeb5f4ca9aa645dc1"},{"sha":"2dd7f93080ee882afcc2aac1a419802a19a668ce","author":{"email":"5baf40bea4aac2d78f51a81218e1428d32fa1839@intel.com","name":"zhichao.li"},"message":"[SPARK-7862] [SQL] Fix the deadlock in script transformation for stderr\n\n[Related PR SPARK-7044] (https://github.com/apache/spark/pull/5671)\n\nAuthor: zhichao.li <zhichao.li@intel.com>\n\nCloses #6404 from zhichao-li/transform and squashes the following commits:\n\n8418c97 [zhichao.li] add comments and remove useless failAfter logic\nd9677e1 [zhichao.li] redirect the error desitination to be the same as the current process","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/2dd7f93080ee882afcc2aac1a419802a19a668ce"},{"sha":"e428b3a951377d47aa80d5f26d6bab979e72e8ab","author":{"email":"808d31295a27ded55339ed818ba0b1912ca15edb@guavus.com","name":"Yash Datta"},"message":"[SPARK-6566] [SQL] Related changes for newer parquet version\n\nThis brings in major improvement in that footers are not read on the driver. This also cleans up the code in parquetTableOperations, where we had to override getSplits to eliminate multiple listStatus calls.\n\ncc liancheng\n\nare there any other changes we need for this ?\n\nAuthor: Yash Datta <Yash.Datta@guavus.com>\n\nCloses #5889 from saucam/parquet_1.6 and squashes the following commits:\n\nd1bf41e [Yash Datta] SPARK-7340: Fix scalastyle and incorporate review comments\nc9aa042 [Yash Datta] SPARK-7340: Use the new user defined filter predicate for pushing down inset into parquet\n56bc750 [Yash Datta] SPARK-7340: Change parquet version to latest release","distinct":true,"url":"https://api.github.com/repos/emptyskull/spark/commits/e428b3a951377d47aa80d5f26d6bab979e72e8ab"}]}