{"action":"opened","issue":{"url":"https://api.github.com/repos/certtools/intelmq/issues/351","labels_url":"https://api.github.com/repos/certtools/intelmq/issues/351/labels{/name}","comments_url":"https://api.github.com/repos/certtools/intelmq/issues/351/comments","events_url":"https://api.github.com/repos/certtools/intelmq/issues/351/events","html_url":"https://github.com/certtools/intelmq/issues/351","id":109806689,"number":351,"title":"BUG: postgresql output bot should not stop on error","user":{"login":"aaronkaplan","id":750019,"avatar_url":"https://avatars.githubusercontent.com/u/750019?v=3","gravatar_id":"","url":"https://api.github.com/users/aaronkaplan","html_url":"https://github.com/aaronkaplan","followers_url":"https://api.github.com/users/aaronkaplan/followers","following_url":"https://api.github.com/users/aaronkaplan/following{/other_user}","gists_url":"https://api.github.com/users/aaronkaplan/gists{/gist_id}","starred_url":"https://api.github.com/users/aaronkaplan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aaronkaplan/subscriptions","organizations_url":"https://api.github.com/users/aaronkaplan/orgs","repos_url":"https://api.github.com/users/aaronkaplan/repos","events_url":"https://api.github.com/users/aaronkaplan/events{/privacy}","received_events_url":"https://api.github.com/users/aaronkaplan/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/certtools/intelmq/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"sebix","id":199050,"avatar_url":"https://avatars.githubusercontent.com/u/199050?v=3","gravatar_id":"","url":"https://api.github.com/users/sebix","html_url":"https://github.com/sebix","followers_url":"https://api.github.com/users/sebix/followers","following_url":"https://api.github.com/users/sebix/following{/other_user}","gists_url":"https://api.github.com/users/sebix/gists{/gist_id}","starred_url":"https://api.github.com/users/sebix/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sebix/subscriptions","organizations_url":"https://api.github.com/users/sebix/orgs","repos_url":"https://api.github.com/users/sebix/repos","events_url":"https://api.github.com/users/sebix/events{/privacy}","received_events_url":"https://api.github.com/users/sebix/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/certtools/intelmq/milestones/2","html_url":"https://github.com/certtools/intelmq/milestones/Release%201","labels_url":"https://api.github.com/repos/certtools/intelmq/milestones/2/labels","id":719078,"number":2,"title":"Release 1","description":"","creator":{"login":"SYNchroACK","id":4184292,"avatar_url":"https://avatars.githubusercontent.com/u/4184292?v=3","gravatar_id":"","url":"https://api.github.com/users/SYNchroACK","html_url":"https://github.com/SYNchroACK","followers_url":"https://api.github.com/users/SYNchroACK/followers","following_url":"https://api.github.com/users/SYNchroACK/following{/other_user}","gists_url":"https://api.github.com/users/SYNchroACK/gists{/gist_id}","starred_url":"https://api.github.com/users/SYNchroACK/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SYNchroACK/subscriptions","organizations_url":"https://api.github.com/users/SYNchroACK/orgs","repos_url":"https://api.github.com/users/SYNchroACK/repos","events_url":"https://api.github.com/users/SYNchroACK/events{/privacy}","received_events_url":"https://api.github.com/users/SYNchroACK/received_events","type":"User","site_admin":false},"open_issues":35,"closed_issues":171,"state":"open","created_at":"2014-07-12T18:19:52Z","updated_at":"2015-10-05T14:21:02Z","due_on":"2014-08-31T07:00:00Z","closed_at":null},"comments":0,"created_at":"2015-10-05T14:21:02Z","updated_at":"2015-10-05T14:21:02Z","closed_at":null,"body":"See issue #350 : when such an error occurs, the postgresql bot stops processing data.\r\nIt should continue processing the other data and not stop entirely.\r\n\r\nFrom the log file:\r\n\r\n```\r\n2015-10-05 15:22:04,062 - postgresql-output - DEBUG - Query: u'INSERT INTO events (\"source.geolocation.latitude\", \"feed.url\", \"source.geolocation.longitude\", \"event_description.text\", \"raw\", \"source.network\", \"source.registry\", \"source.as_name\", \"source.geolocation.city\", \"source.ip\", \"source.abuse_contact\", \"protocol.application\", \"time.observation\", \"source.geolocation.cc\", \"source.allocated\", \"classification.type\", \"source.asn\", \"feed.name\") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)' with values [43.801, u'https://lists.blocklist.de/lists/ssh.txt', 87.6005, u'IP reported as having run attacks on the service SSH', u'MTEwLjE1Mi4xNDUuMTg=', u'110.152.0.0/14', u'apnic', u'CHINANET-BACKBONE No.31,Jin-rong Street,CN', u'\\xdcr\\xfcmqi', u'110.152.145.18', u'cncert@cert.org.cn', u'ssh', u'2015-10-05T13:20:22+00:00', u'CN', u'2009-04-21T00:00:00+00:00', u'ids alert', 4134, u'BlockList.de']\r\n2015-10-05 15:22:04,062 - postgresql-output - ERROR - Bot has found a problem.\r\nTraceback (most recent call last):\r\n  File \"/usr/local/lib/python2.7/dist-packages/intelmq-1.0.0-py2.7.egg/intelmq/lib/bot.py\", line 95, in start\r\n    self.process()\r\n  File \"/usr/local/lib/python2.7/dist-packages/intelmq-1.0.0-py2.7.egg/intelmq/bots/outputs/postgresql/output.py\", line 56, in process\r\n    self.cur.execute(query, values)\r\nUnicodeEncodeError: 'ascii' codec can't encode character u'\\xdc' in position 0: ordinal not in range(128)\r\n```"}}