{"action":"created","issue":{"url":"https://api.github.com/repos/jasonish/py-idstools/issues/19","labels_url":"https://api.github.com/repos/jasonish/py-idstools/issues/19/labels{/name}","comments_url":"https://api.github.com/repos/jasonish/py-idstools/issues/19/comments","events_url":"https://api.github.com/repos/jasonish/py-idstools/issues/19/events","html_url":"https://github.com/jasonish/py-idstools/issues/19","id":92224610,"number":19,"title":"Encoding issue crash","user":{"login":"djtecha","id":3084627,"avatar_url":"https://avatars.githubusercontent.com/u/3084627?v=3","gravatar_id":"","url":"https://api.github.com/users/djtecha","html_url":"https://github.com/djtecha","followers_url":"https://api.github.com/users/djtecha/followers","following_url":"https://api.github.com/users/djtecha/following{/other_user}","gists_url":"https://api.github.com/users/djtecha/gists{/gist_id}","starred_url":"https://api.github.com/users/djtecha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/djtecha/subscriptions","organizations_url":"https://api.github.com/users/djtecha/orgs","repos_url":"https://api.github.com/users/djtecha/repos","events_url":"https://api.github.com/users/djtecha/events{/privacy}","received_events_url":"https://api.github.com/users/djtecha/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-06-30T23:31:25Z","updated_at":"2015-07-01T01:33:57Z","closed_at":null,"body":"Some rule in snort 2.9.7.2-1 is creating a breaking case in the python module:\r\n\r\nINFO: Loaded 25034 rule message map entries.\r\nINFO: Loaded 38 classifications.\r\nTraceback (most recent call last):\r\n  File \"py-idstools/bin/idstools-u2json\", line 12, in <module>\r\n    sys.exit(main())\r\n  File \"/root/py-idstools/idstools/scripts/u2json.py\", line 308, in main\r\n    as_json = json.dumps(formatter.format(record))\r\n  File \"/usr/lib64/python2.7/json/__init__.py\", line 243, in dumps\r\n    return _default_encoder.encode(obj)\r\n  File \"/usr/lib64/python2.7/json/encoder.py\", line 207, in encode\r\n    chunks = self.iterencode(o, _one_shot=True)\r\n  File \"/usr/lib64/python2.7/json/encoder.py\", line 270, in iterencode\r\n    return _iterencode(o, 0)\r\nUnicodeDecodeError: 'utf8' codec can't decode byte 0xec in position 633: invalid continuation byte\r\n\r\n\r\nUsing the latest bits. If I clear out the unified log and restart it it will work until this sequence is hit again."},"comment":{"url":"https://api.github.com/repos/jasonish/py-idstools/issues/comments/117387215","html_url":"https://github.com/jasonish/py-idstools/issues/19#issuecomment-117387215","issue_url":"https://api.github.com/repos/jasonish/py-idstools/issues/19","id":117387215,"user":{"login":"jasonish","id":644115,"avatar_url":"https://avatars.githubusercontent.com/u/644115?v=3","gravatar_id":"","url":"https://api.github.com/users/jasonish","html_url":"https://github.com/jasonish","followers_url":"https://api.github.com/users/jasonish/followers","following_url":"https://api.github.com/users/jasonish/following{/other_user}","gists_url":"https://api.github.com/users/jasonish/gists{/gist_id}","starred_url":"https://api.github.com/users/jasonish/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jasonish/subscriptions","organizations_url":"https://api.github.com/users/jasonish/orgs","repos_url":"https://api.github.com/users/jasonish/repos","events_url":"https://api.github.com/users/jasonish/events{/privacy}","received_events_url":"https://api.github.com/users/jasonish/received_events","type":"User","site_admin":false},"created_at":"2015-07-01T01:33:57Z","updated_at":"2015-07-01T01:33:57Z","body":"If using master from git, pull in commit 67a0dbbc587ed144c0e1029e54f62f51c8ae7ed8 which will log an error message with the failing record to the error stream, the continue processing the log.\r\n\r\nI thought I had handled all encoding issues.  Any chance you can share a failing log file?"}}