{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/7209","labels_url":"https://api.github.com/repos/apache/spark/issues/7209/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/7209/comments","events_url":"https://api.github.com/repos/apache/spark/issues/7209/events","html_url":"https://github.com/apache/spark/pull/7209","id":92819284,"number":7209,"title":"[SPARK-8811][SQL] Read array struct data from parquet error","user":{"login":"Sephiroth-Lin","id":10866770,"avatar_url":"https://avatars.githubusercontent.com/u/10866770?v=3","gravatar_id":"","url":"https://api.github.com/users/Sephiroth-Lin","html_url":"https://github.com/Sephiroth-Lin","followers_url":"https://api.github.com/users/Sephiroth-Lin/followers","following_url":"https://api.github.com/users/Sephiroth-Lin/following{/other_user}","gists_url":"https://api.github.com/users/Sephiroth-Lin/gists{/gist_id}","starred_url":"https://api.github.com/users/Sephiroth-Lin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Sephiroth-Lin/subscriptions","organizations_url":"https://api.github.com/users/Sephiroth-Lin/orgs","repos_url":"https://api.github.com/users/Sephiroth-Lin/repos","events_url":"https://api.github.com/users/Sephiroth-Lin/events{/privacy}","received_events_url":"https://api.github.com/users/Sephiroth-Lin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":26,"created_at":"2015-07-03T08:13:39Z","updated_at":"2015-07-07T06:15:28Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/7209","html_url":"https://github.com/apache/spark/pull/7209","diff_url":"https://github.com/apache/spark/pull/7209.diff","patch_url":"https://github.com/apache/spark/pull/7209.patch"},"body":"JIRA:https://issues.apache.org/jira/browse/SPARK-8811\r\n\r\n```\r\nwe have a table: \r\nt1(c1 string, \r\n   c2 string, \r\n   arr_c1 array<struct<in_c1 string, in_c2 string>>, \r\n   arr_c2 array<struct<in_c1 string, in_c2 string>>\r\n)\r\n\r\nwe save data in parquet.\r\n\r\nfor select * from t1, we know in parquet the fileSchema may be:\r\nmessage hive_schema {\r\n  optional binary c1;\r\n  optional binary c2;\r\n  optional group arr_c1 (LIST) {\r\n    repeated group bag {\r\n      optional group array_element {\r\n        optional binary IN_C1;\r\n        optional binary IN_C2;\r\n      }\r\n    }\r\n  }\r\n  optional group arr_c2 (LIST) {\r\n    repeated group bag {\r\n      optional group array_element {\r\n        optional binary IN_C1;\r\n        optional binary IN_C2;\r\n      }\r\n    }\r\n  }\r\n}\r\nbut the requestSchema is:\r\nmessage root {\r\n  optional binary c1;\r\n  optional binary c2;\r\n  optional group arr_c1 (LIST) {\r\n    repeated group bag {\r\n      optional group element {\r\n        optional binary IN_C1;\r\n        optional binary IN_C2;\r\n      }\r\n    }\r\n  }\r\n  optional group arr_c2 (LIST) {\r\n    repeated group bag {\r\n      optional group element {\r\n        optional binary IN_C1;\r\n        optional binary IN_C2;\r\n      }\r\n    }\r\n  }\r\n}\r\n```\r\nso when read data from parquet will cause java.lang.ArrayIndexOutOfBoundsException"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/119083697","html_url":"https://github.com/apache/spark/pull/7209#issuecomment-119083697","issue_url":"https://api.github.com/repos/apache/spark/issues/7209","id":119083697,"user":{"login":"DoingDone9","id":7529754,"avatar_url":"https://avatars.githubusercontent.com/u/7529754?v=3","gravatar_id":"","url":"https://api.github.com/users/DoingDone9","html_url":"https://github.com/DoingDone9","followers_url":"https://api.github.com/users/DoingDone9/followers","following_url":"https://api.github.com/users/DoingDone9/following{/other_user}","gists_url":"https://api.github.com/users/DoingDone9/gists{/gist_id}","starred_url":"https://api.github.com/users/DoingDone9/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DoingDone9/subscriptions","organizations_url":"https://api.github.com/users/DoingDone9/orgs","repos_url":"https://api.github.com/users/DoingDone9/repos","events_url":"https://api.github.com/users/DoingDone9/events{/privacy}","received_events_url":"https://api.github.com/users/DoingDone9/received_events","type":"User","site_admin":false},"created_at":"2015-07-07T06:15:28Z","updated_at":"2015-07-07T06:15:28Z","body":"LGTM"}}