{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/logstash-forwarder/issues/438","labels_url":"https://api.github.com/repos/elastic/logstash-forwarder/issues/438/labels{/name}","comments_url":"https://api.github.com/repos/elastic/logstash-forwarder/issues/438/comments","events_url":"https://api.github.com/repos/elastic/logstash-forwarder/issues/438/events","html_url":"https://github.com/elastic/logstash-forwarder/issues/438","id":68698926,"number":438,"title":"Read error looking ack & i/o timeout (since 0.4)","user":{"login":"knalli","id":65389,"avatar_url":"https://avatars.githubusercontent.com/u/65389?v=3","gravatar_id":"","url":"https://api.github.com/users/knalli","html_url":"https://github.com/knalli","followers_url":"https://api.github.com/users/knalli/followers","following_url":"https://api.github.com/users/knalli/following{/other_user}","gists_url":"https://api.github.com/users/knalli/gists{/gist_id}","starred_url":"https://api.github.com/users/knalli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/knalli/subscriptions","organizations_url":"https://api.github.com/users/knalli/orgs","repos_url":"https://api.github.com/users/knalli/repos","events_url":"https://api.github.com/users/knalli/events{/privacy}","received_events_url":"https://api.github.com/users/knalli/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":37,"created_at":"2015-04-15T14:36:17Z","updated_at":"2015-04-29T01:08:24Z","closed_at":null,"body":"I'm faced with both `Read error looking for ack: read tcp $host:$port: i/o timeout` and `Failed to tls handshake with $host read tcp $host:$ip: i/o timeout` since a couple of day.\r\n\r\n[The affective result is I'm loosing the events (read: no output in logstash.stdout due debug): no more events on LS side after some seconds/minutes. Restarting LS does not help any more, and the stop of LS even will not made in time (eventually a LS issue itself?). I also notice sometimes a high cpu usage of the logstash daemon (100%, full load on a tiny VM). It also looks there is been a huge memory issue.]\r\n\r\nRunning a LS 1.5RC2, the last changes since last week were:\r\n* update logstash-forwarder from 0.3 to latest 0.4\r\n* activate (aka using) two filters (geoip, useragent)\r\n\r\nNeither disabling the new filters nor reduction to only one active logstash-forwarder agent did help anything... and then I've re-installed the older logstash-forwarder 0.3: The flow of events is being processed successfully again. If I start a logstash-forwarder 0.4 in addition (another \"app host\"), all logstash-forwards (incl. the 0.3) have been the same issue. If I restart all services again except the logstash-forwarder 0.4, it will work again without any issues.\r\n\r\n*So tl;dr*: Only one forwarder agent (0.4) seems to break all other agents (regardless 0.3 or 0.4).\r\n\r\nSite notes: \r\n* All machines are running the latest\r\n** Amazon AMI with Ruby 2p864, Oracle Java 8 40\r\n** Elasticsearch 1.4.4, Logstash 1.5 RC2\r\n* the list of installed plugins (/opt/logstash/bin/plugin --installed --verbose) is exactly the same\r\n\r\n\r\nMy config looks equal in both versions\r\n\r\n```\r\n{\r\n  \"network\": {\r\n    \"servers\": [ \"$host:$port\" ],\r\n    \"timeout\": 15,\r\n    \"ssl ca\": \"/etc/pki/tls/certs/$cert\"\r\n  },\r\n  \"files\": [ {\r\n...    list of about 10 path-type-items with exact file patterns (no wildcards)\r\n  } ]\r\n}\r\n```\r\n\r\nThere are already related issues, but I'm not aware of relevancy to logstash-forwarder 0.4: #421, #429, #431, #91\r\n\r\nLet me know which additional information you need. Because the forwarder was making the trouble, I've created the issue at this place. I'm quite not sure which component is responsible.."},"comment":{"url":"https://api.github.com/repos/elastic/logstash-forwarder/issues/comments/97278048","html_url":"https://github.com/elastic/logstash-forwarder/issues/438#issuecomment-97278048","issue_url":"https://api.github.com/repos/elastic/logstash-forwarder/issues/438","id":97278048,"user":{"login":"afzal273","id":8137882,"avatar_url":"https://avatars.githubusercontent.com/u/8137882?v=3","gravatar_id":"","url":"https://api.github.com/users/afzal273","html_url":"https://github.com/afzal273","followers_url":"https://api.github.com/users/afzal273/followers","following_url":"https://api.github.com/users/afzal273/following{/other_user}","gists_url":"https://api.github.com/users/afzal273/gists{/gist_id}","starred_url":"https://api.github.com/users/afzal273/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/afzal273/subscriptions","organizations_url":"https://api.github.com/users/afzal273/orgs","repos_url":"https://api.github.com/users/afzal273/repos","events_url":"https://api.github.com/users/afzal273/events{/privacy}","received_events_url":"https://api.github.com/users/afzal273/received_events","type":"User","site_admin":false},"created_at":"2015-04-29T01:08:24Z","updated_at":"2015-04-29T01:08:24Z","body":"Lots of times I see this -\r\n\r\n\r\nthostname\t \t%{syslog_hostname}\r\ntmessage\t \t%{syslog_message}\r\ntsyslog_facility\t \tuser-level\r\n#syslog_facility_code\t \t1\r\ntsyslog_severity\t \tnotice\r\n#syslog_severity_code\t \t5\r\n?tags\t \t  _grokparsefailure\r\n\r\nDoes that mean my logstash is overloaded? Because from the same host there are logs which are both parse failure and some that work."}}