{"action":"opened","issue":{"url":"https://api.github.com/repos/logstash-plugins/logstash-output-influxdb/issues/14","labels_url":"https://api.github.com/repos/logstash-plugins/logstash-output-influxdb/issues/14/labels{/name}","comments_url":"https://api.github.com/repos/logstash-plugins/logstash-output-influxdb/issues/14/comments","events_url":"https://api.github.com/repos/logstash-plugins/logstash-output-influxdb/issues/14/events","html_url":"https://github.com/logstash-plugins/logstash-output-influxdb/issues/14","id":64369953,"number":14,"title":"Flush retrying (InfluxDB offline) causes duplicate records in InfluxDB","user":{"login":"wblakecaldwell","id":327741,"avatar_url":"https://avatars.githubusercontent.com/u/327741?v=3","gravatar_id":"","url":"https://api.github.com/users/wblakecaldwell","html_url":"https://github.com/wblakecaldwell","followers_url":"https://api.github.com/users/wblakecaldwell/followers","following_url":"https://api.github.com/users/wblakecaldwell/following{/other_user}","gists_url":"https://api.github.com/users/wblakecaldwell/gists{/gist_id}","starred_url":"https://api.github.com/users/wblakecaldwell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wblakecaldwell/subscriptions","organizations_url":"https://api.github.com/users/wblakecaldwell/orgs","repos_url":"https://api.github.com/users/wblakecaldwell/repos","events_url":"https://api.github.com/users/wblakecaldwell/events{/privacy}","received_events_url":"https://api.github.com/users/wblakecaldwell/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-03-25T20:52:44Z","updated_at":"2015-03-25T20:52:44Z","closed_at":null,"body":"The flush(events) method groups events with the same series and column\r\nnames. This was implemented in a way that modified the input collection.\r\nThis caused an issue with duplicate records being delivered to InfluxDB\r\nbecause flush(events) is called every N seconds (default 1) with the\r\nsame collection until the delivery succeeds. Each loop iteration added\r\nall of the events' points to the first event with the matching series\r\nand column names. By the time flush(events) works - that is, InfluxDB is\r\nback online, that request will have a bunch of duplicate records -\r\nN*(batch size). If the records don't have \"sequence_number\" columns,\r\nthen InfluxDB accepts each duplicate as its own data point, causing a\r\nhuge spike at the moment that delivery resumes.\r\n\r\nhttps://github.com/logstash-plugins/logstash-output-influxdb/blob/master/lib/logstash/outputs/influxdb.rb#L175-L182\r\n"}}