{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/4729","labels_url":"https://api.github.com/repos/apache/spark/issues/4729/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/4729/comments","events_url":"https://api.github.com/repos/apache/spark/issues/4729/events","html_url":"https://github.com/apache/spark/pull/4729","id":58617825,"number":4729,"title":"[SPARK-5950][SQL] Enable inserting array into Hive table saved as Parquet using DataSource API","user":{"login":"viirya","id":68855,"avatar_url":"https://avatars.githubusercontent.com/u/68855?v=3","gravatar_id":"","url":"https://api.github.com/users/viirya","html_url":"https://github.com/viirya","followers_url":"https://api.github.com/users/viirya/followers","following_url":"https://api.github.com/users/viirya/following{/other_user}","gists_url":"https://api.github.com/users/viirya/gists{/gist_id}","starred_url":"https://api.github.com/users/viirya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/viirya/subscriptions","organizations_url":"https://api.github.com/users/viirya/orgs","repos_url":"https://api.github.com/users/viirya/repos","events_url":"https://api.github.com/users/viirya/events{/privacy}","received_events_url":"https://api.github.com/users/viirya/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":19,"created_at":"2015-02-23T17:20:07Z","updated_at":"2015-02-27T00:13:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/4729","html_url":"https://github.com/apache/spark/pull/4729","diff_url":"https://github.com/apache/spark/pull/4729.diff","patch_url":"https://github.com/apache/spark/pull/4729.patch"},"body":"Currently `ParquetConversions` in `HiveMetastoreCatalog` does not really work. One reason is that table is not part of the children nodes of `InsertIntoTable`. So the replacing is not working.\r\n\r\nWhen we create a Parquet table in Hive with ARRAY field. In default `ArrayType` has `containsNull` as true. It affects the table's schema. But when inserting data into the table later, the schema of inserting data can be  with `containsNull` as true or false. That makes the inserting/reading failed.\r\n\r\nA similar problem is reported in https://issues.apache.org/jira/browse/SPARK-5508.\r\n\r\nHive seems only support null elements array. So this pr enables same behavior.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/76304040","html_url":"https://github.com/apache/spark/pull/4729#issuecomment-76304040","issue_url":"https://api.github.com/repos/apache/spark/issues/4729","id":76304040,"user":{"login":"viirya","id":68855,"avatar_url":"https://avatars.githubusercontent.com/u/68855?v=3","gravatar_id":"","url":"https://api.github.com/users/viirya","html_url":"https://github.com/viirya","followers_url":"https://api.github.com/users/viirya/followers","following_url":"https://api.github.com/users/viirya/following{/other_user}","gists_url":"https://api.github.com/users/viirya/gists{/gist_id}","starred_url":"https://api.github.com/users/viirya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/viirya/subscriptions","organizations_url":"https://api.github.com/users/viirya/orgs","repos_url":"https://api.github.com/users/viirya/repos","events_url":"https://api.github.com/users/viirya/events{/privacy}","received_events_url":"https://api.github.com/users/viirya/received_events","type":"User","site_admin":false},"created_at":"2015-02-27T00:13:18Z","updated_at":"2015-02-27T00:13:18Z","body":"In fact, even #4782 doesn't solve the issue I reported in this pr. The unit test is failed before hitting the data insertion issue..."}}