{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/4562","labels_url":"https://api.github.com/repos/apache/spark/issues/4562/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/4562/comments","events_url":"https://api.github.com/repos/apache/spark/issues/4562/events","html_url":"https://github.com/apache/spark/pull/4562","id":57436207,"number":4562,"title":"[SPARK-5852] [SQL] Passdown the schema for Parquet File in HiveContext","user":{"login":"chenghao-intel","id":2367031,"avatar_url":"https://avatars.githubusercontent.com/u/2367031?v=3","gravatar_id":"","url":"https://api.github.com/users/chenghao-intel","html_url":"https://github.com/chenghao-intel","followers_url":"https://api.github.com/users/chenghao-intel/followers","following_url":"https://api.github.com/users/chenghao-intel/following{/other_user}","gists_url":"https://api.github.com/users/chenghao-intel/gists{/gist_id}","starred_url":"https://api.github.com/users/chenghao-intel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chenghao-intel/subscriptions","organizations_url":"https://api.github.com/users/chenghao-intel/orgs","repos_url":"https://api.github.com/users/chenghao-intel/repos","events_url":"https://api.github.com/users/chenghao-intel/events{/privacy}","received_events_url":"https://api.github.com/users/chenghao-intel/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-02-12T09:29:44Z","updated_at":"2015-02-17T07:27:58Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/4562","html_url":"https://github.com/apache/spark/pull/4562","diff_url":"https://github.com/apache/spark/pull/4562.diff","patch_url":"https://github.com/apache/spark/pull/4562.patch"},"body":"It's not allowed to be the empty directory for parquet, for example, it will failed when query the following\r\n```\r\nCREATE TABLE parquet_test (id int, str string) STORED AS PARQUET;\r\nSELECT * FROM parquet_test;\r\n```\r\n\r\nIt throws exception like:\r\n```\r\njava.lang.UnsupportedOperationException: empty.reduceLeft\r\n\tat scala.collection.TraversableOnce$class.reduceLeft(TraversableOnce.scala:167)\r\n\tat scala.collection.mutable.ArrayBuffer.scala$collection$IndexedSeqOptimized$$super$reduceLeft(ArrayBuffer.scala:47)\r\n\tat scala.collection.IndexedSeqOptimized$class.reduceLeft(IndexedSeqOptimized.scala:68)\r\n\tat scala.collection.mutable.ArrayBuffer.reduceLeft(ArrayBuffer.scala:47)\r\n\tat scala.collection.TraversableOnce$class.reduce(TraversableOnce.scala:195)\r\n\tat scala.collection.AbstractTraversable.reduce(Traversable.scala:105)\r\n\tat org.apache.spark.sql.parquet.ParquetRelation2$.readSchema(newParquet.scala:633)\r\n\tat org.apache.spark.sql.parquet.ParquetRelation2$MetadataCache.org$apache$spark$sql$parquet$ParquetRelation2$MetadataCache$$readSchema(newParquet.scala:349)\r\n\tat org.apache.spark.sql.parquet.ParquetRelation2$MetadataCache$$anonfun$refresh$8.apply(newParquet.scala:290)\r\n\tat org.apache.spark.sql.parquet.ParquetRelation2$MetadataCache$$anonfun$refresh$8.apply(newParquet.scala:290)\r\n\tat scala.Option.getOrElse(Option.scala:120)\r\n\tat org.apache.spark.sql.parquet.ParquetRelation2$MetadataCache.refresh(newParquet.scala:290)\r\n\tat org.apache.spark.sql.parquet.ParquetRelation2.<init>(newParquet.scala:354)\r\n\tat org.apache.spark.sql.hive.HiveMetastoreCatalog.org$apache$spark$sql$hive$HiveMetastoreCatalog$$convertToParquetRelation(HiveMetastoreCatalog.scala:218)\r\n\tat org.apache.spark.sql.hive.HiveMetastoreCatalog$ParquetConversions$$anonfun$apply$4.apply(HiveMetastoreCatalog.scala:446)\r\n\tat org.apache.spark.sql.hive.HiveMetastoreCatalog$ParquetConversions$$anonfun$apply$4.apply(HiveMetastoreCatalog.scala:445)\r\n\tat scala.collection.IndexedSeqOptimized$class.foldl(IndexedSeqOptimized.scala:51)\r\n\tat scala.collection.IndexedSeqOptimized$class.foldLeft(IndexedSeqOptimized.scala:60)\r\n\tat scala.collection.mutable.ArrayBuffer.foldLeft(ArrayBuffer.scala:47)\r\n\tat org.apache.spark.sql.hive.HiveMetastoreCatalog$ParquetConversions$.apply(HiveMetastoreCatalog.scala:445)\r\n\tat org.apache.spark.sql.hive.HiveMetastoreCatalog$ParquetConversions$.apply(HiveMetastoreCatalog.scala:422)\r\n\tat org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:61)\r\n\tat org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1$$anonfun$apply$2.apply(RuleExecutor.scala:59)\r\n\tat scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:111)\r\n\tat scala.collection.immutable.List.foldLeft(List.scala:84)\r\n\tat org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:59)\r\n\tat org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$apply$1.apply(RuleExecutor.scala:51)\r\n\tat scala.collection.immutable.List.foreach(List.scala:318)\r\n\tat org.apache.spark.sql.catalyst.rules.RuleExecutor.apply(RuleExecutor.scala:51)\r\n\tat org.apache.spark.sql.SQLContext$QueryExecution.analyzed$lzycompute(SQLContext.scala:917)\r\n\tat org.apache.spark.sql.SQLContext$QueryExecution.analyzed(SQLContext.scala:917)\r\n\tat org.apache.spark.sql.DataFrameImpl.<init>(DataFrameImpl.scala:61)\r\n\tat org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:35)\r\n\tat org.apache.spark.sql.hive.HiveContext.sql(HiveContext.scala:77)\r\n\tat org.apache.spark.sql.hive.thriftserver.AbstractSparkSQLDriver.run(AbstractSparkSQLDriver.scala:57)\r\n\tat org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.processCmd(SparkSQLCLIDriver.scala:275)\r\n\tat org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)\r\n\tat org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:211)\r\n\tat org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)\r\n```"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/74627418","html_url":"https://github.com/apache/spark/pull/4562#issuecomment-74627418","issue_url":"https://api.github.com/repos/apache/spark/issues/4562","id":74627418,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2015-02-17T07:27:58Z","updated_at":"2015-02-17T07:27:58Z","body":"  [Test build #27615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27615/consoleFull) for   PR 4562 at commit [`36978d1`](https://github.com/apache/spark/commit/36978d1835ab6e0266ad3787b33056b573fd59e8).\n * This patch merges cleanly."}}