{"action":"created","issue":{"url":"https://api.github.com/repos/HuaweiBigData/astro/issues/45","labels_url":"https://api.github.com/repos/HuaweiBigData/astro/issues/45/labels{/name}","comments_url":"https://api.github.com/repos/HuaweiBigData/astro/issues/45/comments","events_url":"https://api.github.com/repos/HuaweiBigData/astro/issues/45/events","html_url":"https://github.com/HuaweiBigData/astro/pull/45","id":118585445,"number":45,"title":"Access hbase tables from hive metadata. And support pre-loded hbase tables.","user":{"login":"ravipesala","id":8134086,"avatar_url":"https://avatars.githubusercontent.com/u/8134086?v=3","gravatar_id":"","url":"https://api.github.com/users/ravipesala","html_url":"https://github.com/ravipesala","followers_url":"https://api.github.com/users/ravipesala/followers","following_url":"https://api.github.com/users/ravipesala/following{/other_user}","gists_url":"https://api.github.com/users/ravipesala/gists{/gist_id}","starred_url":"https://api.github.com/users/ravipesala/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ravipesala/subscriptions","organizations_url":"https://api.github.com/users/ravipesala/orgs","repos_url":"https://api.github.com/users/ravipesala/repos","events_url":"https://api.github.com/users/ravipesala/events{/privacy}","received_events_url":"https://api.github.com/users/ravipesala/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-11-24T11:17:43Z","updated_at":"2015-11-24T11:46:55Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/HuaweiBigData/astro/pulls/45","html_url":"https://github.com/HuaweiBigData/astro/pull/45","diff_url":"https://github.com/HuaweiBigData/astro/pull/45.diff","patch_url":"https://github.com/HuaweiBigData/astro/pull/45.patch"},"body":"1. Accessing hive tables with HbaseStorageHandler. \r\n If the hive table is created with HbaseStorageHandler like below.\r\n```\r\nCREATE EXTERNAL TABLE TestHiveTable(\r\n  strcol STRING,\r\n  longcol BIGINT,\r\n  floatcol FLOAT,\r\n  doublecol DOUBLE )\r\n stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'\r\n WITH SERDEPROPERTIES (\r\n \"hbase.columns.mapping\" = \"cf1:hstrcol#b,cf2:hlongcol#b,cf2:hfloatcol#b,:key#b\" )\r\n TBLPROPERTIES (\"hbase.table.name\" = \"TestHbaseTable\")\r\n```\r\nUser can access this table directly in Astro and queries the data from Astro.\r\nLimitations  : \r\n1.Key column should be single column, not composite columns.\r\n2.The data in Hbase table should be loaded with Hbase Bytes utility.\r\n3.The hive columns should be binary format.\r\n4.Not yet supported the following datatypes structs,map,array,union,timestamp,date .\r\n5.Only it supports the SparkSQL standard SQL.\r\n6.It cannot access the hive UDF and UDAF.\r\n\r\n2. Supporting pre-loaded Hbase table in Astro.\r\nUser can create the Astro table with pre-loaded hbase as follows.\r\n```\r\nCREATE TABLE TestPreLoadedTable(\r\n  strcol STRING,\r\n  longcol LONG,\r\n  floatcol FLOAT,\r\n  doublecol DOUBLE\r\n)\r\nUSING org.apache.spark.sql.hbase.HBaseSource\r\nOPTIONS(\r\n  hbaseTableName \"TestHbaseTable\",\r\n  keyCols \"doublecol\",\r\n  colsMapping \"longcol=cf2.hlongcol, floatcol=cf2.hfloatcol, strcol=cf1.hstrcol, intcol=cf2.hintcol\",\r\n  encodingFormat \"hbasebinaryformat\"\r\n)\r\n```\r\nWhile creating the table, user can mention the ```encodingFormat``` as ```hbasebinaryformat```, so that Astro uses Hbase Bytes utility to read the data from Hbase.\r\n\r\nLimitations : \r\n1.Key column should be single column, if user wants it composite then it should comply the Astro keyformat(simply concatenating the values together, with a zero byte character used as a separator after a variable length type).\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/HuaweiBigData/astro/issues/comments/159241897","html_url":"https://github.com/HuaweiBigData/astro/pull/45#issuecomment-159241897","issue_url":"https://api.github.com/repos/HuaweiBigData/astro/issues/45","id":159241897,"user":{"login":"ravipesala","id":8134086,"avatar_url":"https://avatars.githubusercontent.com/u/8134086?v=3","gravatar_id":"","url":"https://api.github.com/users/ravipesala","html_url":"https://github.com/ravipesala","followers_url":"https://api.github.com/users/ravipesala/followers","following_url":"https://api.github.com/users/ravipesala/following{/other_user}","gists_url":"https://api.github.com/users/ravipesala/gists{/gist_id}","starred_url":"https://api.github.com/users/ravipesala/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ravipesala/subscriptions","organizations_url":"https://api.github.com/users/ravipesala/orgs","repos_url":"https://api.github.com/users/ravipesala/repos","events_url":"https://api.github.com/users/ravipesala/events{/privacy}","received_events_url":"https://api.github.com/users/ravipesala/received_events","type":"User","site_admin":false},"created_at":"2015-11-24T11:46:55Z","updated_at":"2015-11-24T11:46:55Z","body":"retest this please"}}