{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/5733","labels_url":"https://api.github.com/repos/apache/spark/issues/5733/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/5733/comments","events_url":"https://api.github.com/repos/apache/spark/issues/5733/events","html_url":"https://github.com/apache/spark/pull/5733","id":71481935,"number":5733,"title":"[SPARK-6923][SPARK-7550][SQL] Hive MetaStore API cannot access Data Sourced table schema correctly","user":{"login":"chenghao-intel","id":2367031,"avatar_url":"https://avatars.githubusercontent.com/u/2367031?v=3","gravatar_id":"","url":"https://api.github.com/users/chenghao-intel","html_url":"https://github.com/chenghao-intel","followers_url":"https://api.github.com/users/chenghao-intel/followers","following_url":"https://api.github.com/users/chenghao-intel/following{/other_user}","gists_url":"https://api.github.com/users/chenghao-intel/gists{/gist_id}","starred_url":"https://api.github.com/users/chenghao-intel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chenghao-intel/subscriptions","organizations_url":"https://api.github.com/users/chenghao-intel/orgs","repos_url":"https://api.github.com/users/chenghao-intel/repos","events_url":"https://api.github.com/users/chenghao-intel/events{/privacy}","received_events_url":"https://api.github.com/users/chenghao-intel/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":38,"created_at":"2015-04-28T05:22:02Z","updated_at":"2015-07-29T05:18:00Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/5733","html_url":"https://github.com/apache/spark/pull/5733","diff_url":"https://github.com/apache/spark/pull/5733.diff","patch_url":"https://github.com/apache/spark/pull/5733.patch"},"body":"Writing the dummy information into hive metastore for DataSourced table of DataFrame, so Hive could read the spark sql backend table. \r\n\r\nWe can disable this feature by \r\n```\r\nset spark.sql.hive.writeDataSourceSchema=false;\r\n```\r\n\r\nKnown issues:\r\n- As parquet in Spark SQL is based on 1.7.0, but Hive is supposed to be 1.3.2, it's will cause exception for Hive to `read` a Spark SQL parquet data sourced table.\r\n- JSON / JDBC data source is not a built-in function for Hive, we will not handle it either.\r\n- Spark SQL probably not effect if user alter the table schema via Hive.\r\n\r\nIn long term, as @liancheng suggested, we'd better provide a Hive StoregeHandler to bridge the Spark SQL data source."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/125846725","html_url":"https://github.com/apache/spark/pull/5733#issuecomment-125846725","issue_url":"https://api.github.com/repos/apache/spark/issues/5733","id":125846725,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-07-29T05:18:00Z","updated_at":"2015-07-29T05:18:00Z","body":"Merged build started."}}