{"action":"created","issue":{"url":"https://api.github.com/repos/SAP/PyRFC/issues/4","labels_url":"https://api.github.com/repos/SAP/PyRFC/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/SAP/PyRFC/issues/4/comments","events_url":"https://api.github.com/repos/SAP/PyRFC/issues/4/events","html_url":"https://github.com/SAP/PyRFC/issues/4","id":43797714,"number":4,"title":"UnicodeDecodeError: 'utf8' codec can't decode byte 0xc2","user":{"login":"martinduffy","id":5431249,"avatar_url":"https://avatars.githubusercontent.com/u/5431249?v=3","gravatar_id":"","url":"https://api.github.com/users/martinduffy","html_url":"https://github.com/martinduffy","followers_url":"https://api.github.com/users/martinduffy/followers","following_url":"https://api.github.com/users/martinduffy/following{/other_user}","gists_url":"https://api.github.com/users/martinduffy/gists{/gist_id}","starred_url":"https://api.github.com/users/martinduffy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/martinduffy/subscriptions","organizations_url":"https://api.github.com/users/martinduffy/orgs","repos_url":"https://api.github.com/users/martinduffy/repos","events_url":"https://api.github.com/users/martinduffy/events{/privacy}","received_events_url":"https://api.github.com/users/martinduffy/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2014-09-24T18:21:18Z","updated_at":"2015-03-13T11:14:16Z","closed_at":"2015-03-13T11:14:16Z","body":"I am developing a query for the HRP1000 table using RFC_READ_TABLE. The code is below. I have used this same code for a number of other tables with no issue. I am querying for the Position Codes and Position Descriptions in HRP1000.\r\n\r\nI get the error below before getting all of the rows in the table that satisfy the query. The error is happening during the call for the RFC_READ_TABLE by PyRFC.\r\n\r\nI am not sure if there is anything I could do to trap this error. Any suggestion would be appreciated. I am thinking that the fix is going to require some modification to PyRFC. I tried using the unicodecsv module for python but that will not help I think.\r\n\r\nLet me mention that though the code that I have included here is in development it works really well to get data extracts from SAP and written to a csv file and could be pretty easily changed for other formats of files and includes using all or almost all of the parameters and options for RFC_READ_TABLE.\r\n\r\n=============  Error  ===================================\r\nTraceback (most recent call last):\r\n  File \"I:\\Apps\\PyRFC-master\\examples\\sap-hrp1000-p.py\", line 167, in <module>\r\n    main()\r\n  File \"I:\\Apps\\PyRFC-master\\examples\\sap-hrp1000-p.py\", line 65, in main\r\n    tables = conn.call('RFC_READ_TABLE', QUERY_TABLE=table, OPTIONS = options, DELIMITER='|', FIELDS = fields2, ROWSKIPS = rowskips,ROWCOUNT = ROWS_AT_A_TIME)\r\n  File \"_pyrfc.pyx\", line 368, in pyrfc._pyrfc.Connection.call (pyrfc\\_pyrfc.c:4006)\r\n  File \"_pyrfc.pyx\", line 1837, in pyrfc._pyrfc.wrapResult (pyrfc\\_pyrfc.c:19400)\r\n  File \"_pyrfc.pyx\", line 1910, in pyrfc._pyrfc.wrapVariable (pyrfc\\_pyrfc.c:19954)\r\n  File \"_pyrfc.pyx\", line 1882, in pyrfc._pyrfc.wrapTable (pyrfc\\_pyrfc.c:19732)\r\n  File \"_pyrfc.pyx\", line 1855, in pyrfc._pyrfc.wrapStructure (pyrfc\\_pyrfc.c:19609)\r\n  File \"_pyrfc.pyx\", line 1917, in pyrfc._pyrfc.wrapVariable (pyrfc\\_pyrfc.c:20034)\r\n  File \"_pyrfc.pyx\", line 2049, in pyrfc._pyrfc.wrapString (pyrfc\\_pyrfc.c:21861)\r\n  File \"\\\\ms240xxxx.na.xxxxxx.net\\ctxredir$\\mdufXXXX\\Apps\\python2.7.8\\lib\\encodings\\utf_8.py\", line 16, in decode\r\n    return codecs.utf_8_decode(input, errors, True)\r\nUnicodeDecodeError: 'utf8' codec can't decode byte 0xc2 in position 46: unexpected end of data\r\n\r\n========================== End of Error  ===================\r\n\r\n==================  Code ================================\r\n\r\nfrom pyrfc import Connection, ABAPApplicationError, LogonError\r\nfrom datetime import date\r\nfrom ConfigParser import ConfigParser\r\nimport unicodecsv\r\n \r\nconn = \"\"\r\n\r\nNOW = date.today()    # get today's date\r\nDATESTR = NOW.strftime(\"%m_%d_%Y\")  # Assign formatted date to variable\r\nFILENAME = (\"HRP000_Position_snapshot_\" + DATESTR + \".csv\")\r\n\r\nf = open(FILENAME,'w')\r\n \r\ndef main():\r\n    try:\r\n        config = ConfigParser()\r\n        config.read('sapnwrfc.cfg')\r\n        #params_connection = config._sections['connection']\r\n        params_connection = config._sections['pe1']\r\n#        params_connection[\"user\"] = user\r\n#        params_connection[\"passwd\"] = passwd\r\n        conn = Connection(**params_connection)\r\n    except LogonError:\r\n        print \"Invalid username or password\"\r\n \r\n    fields = []\r\n    fields_name = []\r\n    counter = 0\r\n    table=\"HRP1000\"\r\n    #print table\r\n    \r\n    #fields2 =[{'FIELDNAME':'PLVAR'}, {'FIELDNAME':'OTYPE'}, {'FIELDNAME':'OBJID'}, {'FIELDNAME':'BEGDA'}, {'FIELDNAME':'ENDDA'}, {'FIELDNAME':'OTJID'}, {'FIELDNAME':'SHORT'},{'FIELDNAME':'STEXT'}]\r\n    fields2 =[{'FIELDNAME':'OBJID'}, {'FIELDNAME':'BEGDA'}, {'FIELDNAME':'ENDDA'},{'FIELDNAME':'STEXT'}]\r\n    \r\n\r\n    options = [{ 'TEXT': \"PLVAR = '01' AND OTYPE = 'S' AND ENDDA = '99991231'\"}]\r\n        \r\n    \r\n    ROWS_AT_A_TIME = 100\r\n    rowskips = 0\r\n    count = 0\r\n    while True:\r\n        #print rowskips\r\n        data_fields = []\r\n        data_names = []\r\n        #tables = conn.call('RFC_READ_TABLE', OPTIONS = options, FIELDS = fields2, QUERY_TABLE=table, DELIMITER='|', ROWSKIPS = rowskips,ROWCOUNT = ROWS_AT_A_TIME)\r\n        #tables = conn.call(\"RFC_READ_TABLE\", OPTIONS = options, QUERY_TABLE= table, DELIMITER='|')\r\n        print \"ok before tables\"\r\n        tables = conn.call('RFC_READ_TABLE', QUERY_TABLE=table, OPTIONS = options, DELIMITER='|', FIELDS = fields2, ROWSKIPS = rowskips,ROWCOUNT = ROWS_AT_A_TIME)\r\n\r\n        print tables\r\n        \r\n        print \"ok after tables\"\r\n        data_fields = tables[\"DATA\"]\r\n        data_names = tables[\"FIELDS\"]\r\n        #data_fields = data_fields.encode(\"utf-8\")\r\n\r\n##        print data_fields\r\n##        print \"\"\r\n##        print \"\"\r\n##        print \"\"\r\n##        print data_names\r\n##        print \"\"\r\n##        print \"\"\r\n##        print \"\"\r\n        \r\n        long_fields = len(data_fields)\r\n        long_names = len(data_names)\r\n\r\n        rowskips = rowskips + ROWS_AT_A_TIME\r\n\r\n        #print rowskips\r\n        fields = []\r\n        field_name = []\r\n        \r\n        for line in range(0, long_fields):\r\n            #fields.append(unicodecsv.reader(data_fields[line][\"WA\"].strip(), encoding='utf-8'))\r\n            #fields = (unicodecsv.writer(fields, encoding='utf-8')\r\n        \r\n            fields.append(data_fields[line][\"WA\"].strip())\r\n        #print fields\r\n        #print \"\"\r\n        #print \"\"\r\n            \r\n        for line in range(0, long_names):\r\n            fields_name.append(data_names[line][\"FIELDNAME\"].strip())\r\n\r\n        #print fields_name\r\n        #print \"\"\r\n        #print \"\"\r\n        #print \"\"\r\n        \r\n        output=\"\"    \r\n \r\n        for line in range(0, long_names):\r\n            field_name = fields_name[line]\r\n            output = output + field_name + \",\"\r\n        output = output.rstrip(\",\")\r\n        output = output + \"\\n\"\r\n        if count == 0:\r\n            print output\r\n            f.write(str(output))\r\n            count = 1\r\n        output = \"\"\r\n        \r\n        for line in range(0, long_fields):\r\n            data_split = fields[line].split(\"|\")\r\n            \r\n            for line in range(0, long_names):\r\n                #print data_split[line]\r\n                #print \" before data_split ok\"\r\n                \r\n                output = output + data_split[line].encode(\"utf-8\") + \",\"\r\n                #print \"after data_split ok\" \r\n                #output = unicodecsv.reader(line, encoding='utf-8') \r\n                #output = output + data_split[line].encode(\"utf-8\") + \",\"\r\n                #output = output + data_split[line] + \",\"\r\n                #r = unicodecsv.reader(f, encoding='utf-8')\r\n            output = output.rstrip(\",\")\r\n            output = output + \"\\n\"\r\n        print output\r\n        print \"output ok before write\"\r\n        f.write(str(output))\r\n        print \"output ok after write\"\r\n        output = \"\"\r\n    \r\n        if len(tables['DATA']) < ROWS_AT_A_TIME:\r\n            break\r\n    conn.close()\r\n\r\n    f.close()      \r\n\r\nf2 = open(FILENAME,'r+')\r\nfor line in f2:\r\n    if line.strip():\r\n        f2.write(line)\r\n\r\n                \r\n    #except ABAPApplicationError:\r\n    #    output = \"Table %s was not found\" % table\r\n\r\n#        output = \"<div align='center'><h1>Table %s was not found</h1></div>\" % table\r\n    #    return output\r\n \r\n    ##return output\r\n\r\nf2.close()\r\n\r\n\r\nif __name__ == '__main__':\r\n    main()\r\n"},"comment":{"url":"https://api.github.com/repos/SAP/PyRFC/issues/comments/78923420","html_url":"https://github.com/SAP/PyRFC/issues/4#issuecomment-78923420","issue_url":"https://api.github.com/repos/SAP/PyRFC/issues/4","id":78923420,"user":{"login":"bsrdjan","id":1697847,"avatar_url":"https://avatars.githubusercontent.com/u/1697847?v=3","gravatar_id":"","url":"https://api.github.com/users/bsrdjan","html_url":"https://github.com/bsrdjan","followers_url":"https://api.github.com/users/bsrdjan/followers","following_url":"https://api.github.com/users/bsrdjan/following{/other_user}","gists_url":"https://api.github.com/users/bsrdjan/gists{/gist_id}","starred_url":"https://api.github.com/users/bsrdjan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bsrdjan/subscriptions","organizations_url":"https://api.github.com/users/bsrdjan/orgs","repos_url":"https://api.github.com/users/bsrdjan/repos","events_url":"https://api.github.com/users/bsrdjan/events{/privacy}","received_events_url":"https://api.github.com/users/bsrdjan/received_events","type":"User","site_admin":false},"created_at":"2015-03-13T11:14:16Z","updated_at":"2015-03-13T11:14:16Z","body":"As no comments, looks closed."}}