{"action":"closed","issue":{"url":"https://api.github.com/repos/qubole/qds-sdk-py/issues/94","labels_url":"https://api.github.com/repos/qubole/qds-sdk-py/issues/94/labels{/name}","comments_url":"https://api.github.com/repos/qubole/qds-sdk-py/issues/94/comments","events_url":"https://api.github.com/repos/qubole/qds-sdk-py/issues/94/events","html_url":"https://github.com/qubole/qds-sdk-py/issues/94","id":64402546,"number":94,"title":"Properly reading results from Hive queries in Pandas in Python 3","user":{"login":"amelio-vazquez-reina","id":1993919,"avatar_url":"https://avatars.githubusercontent.com/u/1993919?v=3","gravatar_id":"","url":"https://api.github.com/users/amelio-vazquez-reina","html_url":"https://github.com/amelio-vazquez-reina","followers_url":"https://api.github.com/users/amelio-vazquez-reina/followers","following_url":"https://api.github.com/users/amelio-vazquez-reina/following{/other_user}","gists_url":"https://api.github.com/users/amelio-vazquez-reina/gists{/gist_id}","starred_url":"https://api.github.com/users/amelio-vazquez-reina/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amelio-vazquez-reina/subscriptions","organizations_url":"https://api.github.com/users/amelio-vazquez-reina/orgs","repos_url":"https://api.github.com/users/amelio-vazquez-reina/repos","events_url":"https://api.github.com/users/amelio-vazquez-reina/events{/privacy}","received_events_url":"https://api.github.com/users/amelio-vazquez-reina/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-03-26T00:23:42Z","updated_at":"2015-03-31T20:26:23Z","closed_at":"2015-03-31T20:26:23Z","body":"What is the best way to read the output from disk with Pandas after using `cmd.get_results` ? (e.g. from a Hive command).\r\n\r\nFor example, consider the following:\r\n``` python\r\n\r\nout_file = 'results.csv'\r\ndelimiter = chr(1)\r\n....\r\n\r\nQubole.configure(qubole_key)\r\nhc_params = ['--query', query]\r\nhive_args = HiveCommand.parse(hc_params)\r\ncmd = HiveCommand.run(**hive_args)\r\nif (HiveCommand.is_success(cmd.status)):\r\n  with open(out_file, 'wt') as writer:\r\n  cmd.get_results(writer, delim=delimiter, inline=False)\r\n```\r\n\r\nIf, after successfully running the query, I then inspect the first few bytes of `results.csv`, I see the following:\r\n```\r\n$ head -c 300 results.csv\r\nb'flight_uid\\twinning_price\\tbid_price\\timpressions_source_timestamp\\n'b'0FY6ZsrnMy\\x012000\\x012270.0\\x011427243278000\\n0FamrXG9AW\\x01710\\x01747.0\\x011427243733000\\n0FY6ZsrnMy\\x012000\\x012270.0\\x011427245266000\\n0FY6ZsrnMy\\x012000\\x012270.0\\x011427245088000\\n0FamrXG9AW\\x01330\\x01747.0\\x011427243407000\\n0FamrXG9AW\\x01710\\x01747.0\\x011427243981000\\n0FamrXG9AW\\x01490\\x01747.0\\x011427245289000\\n\r\n```\r\n\r\nWhen I try to open this in [Pandas](http://pandas.pydata.org/):\r\n\r\n`df = pd.read_csv('results.csv')`\r\n\r\nit obviously doesn't work (I get an empty DataFrame), since it isn't properly formatted as a csv file. \r\n\r\nWhile I could try to open `results.csv` and post-process it (to remove `b'`, etc.) before I open it in Pandas, this would be a quite hacky way to load it.\r\n\r\nAm I using the interface correctly? This is using the very last version of `qds_sdk`: `1.4.2` from a three hours ago."}}