{"action":"created","issue":{"url":"https://api.github.com/repos/apache/parquet-mr/issues/210","labels_url":"https://api.github.com/repos/apache/parquet-mr/issues/210/labels{/name}","comments_url":"https://api.github.com/repos/apache/parquet-mr/issues/210/comments","events_url":"https://api.github.com/repos/apache/parquet-mr/issues/210/events","html_url":"https://github.com/apache/parquet-mr/pull/210","id":86460647,"number":210,"title":"PARQUET-304: Add an option to make requested schema case insensitive in read path","user":{"login":"saucam","id":1253893,"avatar_url":"https://avatars.githubusercontent.com/u/1253893?v=3","gravatar_id":"","url":"https://api.github.com/users/saucam","html_url":"https://github.com/saucam","followers_url":"https://api.github.com/users/saucam/followers","following_url":"https://api.github.com/users/saucam/following{/other_user}","gists_url":"https://api.github.com/users/saucam/gists{/gist_id}","starred_url":"https://api.github.com/users/saucam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/saucam/subscriptions","organizations_url":"https://api.github.com/users/saucam/orgs","repos_url":"https://api.github.com/users/saucam/repos","events_url":"https://api.github.com/users/saucam/events{/privacy}","received_events_url":"https://api.github.com/users/saucam/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-06-09T05:46:13Z","updated_at":"2015-06-13T07:41:57Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/parquet-mr/pulls/210","html_url":"https://github.com/apache/parquet-mr/pull/210","diff_url":"https://github.com/apache/parquet-mr/pull/210.diff","patch_url":"https://github.com/apache/parquet-mr/pull/210.patch"},"body":"For projects such as hive and spark-sql which use parquet, the schema of the stored tables is always lowercase (because of limitations of hive metastore). It would be great if we can have a configurable option to read data from parquet irrespective of the case of the requested schema , supplied via ReadContext object.\r\n\r\nThis PR adds configurable option of ParquetInputFormat.CASE_SENSITIVITY which can be set to false. In that case, parquet will resolve requested columns irrespective of case. This alleviates projects like spark-sql to read footers at the driver side and reconcile schema (to be read from footers) with metastore schema before calling parquet read."},"comment":{"url":"https://api.github.com/repos/apache/parquet-mr/issues/comments/111684674","html_url":"https://github.com/apache/parquet-mr/pull/210#issuecomment-111684674","issue_url":"https://api.github.com/repos/apache/parquet-mr/issues/210","id":111684674,"user":{"login":"saucam","id":1253893,"avatar_url":"https://avatars.githubusercontent.com/u/1253893?v=3","gravatar_id":"","url":"https://api.github.com/users/saucam","html_url":"https://github.com/saucam","followers_url":"https://api.github.com/users/saucam/followers","following_url":"https://api.github.com/users/saucam/following{/other_user}","gists_url":"https://api.github.com/users/saucam/gists{/gist_id}","starred_url":"https://api.github.com/users/saucam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/saucam/subscriptions","organizations_url":"https://api.github.com/users/saucam/orgs","repos_url":"https://api.github.com/users/saucam/repos","events_url":"https://api.github.com/users/saucam/events{/privacy}","received_events_url":"https://api.github.com/users/saucam/received_events","type":"User","site_admin":false},"created_at":"2015-06-13T07:41:57Z","updated_at":"2015-06-13T07:41:57Z","body":"@danielcweeks this is possible to do in spark as well (have tried, works for projections) but how do you resolve column names in filter objects which are pushed down in parquet before creating the tasks ? For that it becomes inevitable to have a resolved schema on the driver side ? I wonder if and how hive is able to resolve that ?"}}