{"action":"created","issue":{"url":"https://api.github.com/repos/fluent/fluentd/issues/596","labels_url":"https://api.github.com/repos/fluent/fluentd/issues/596/labels{/name}","comments_url":"https://api.github.com/repos/fluent/fluentd/issues/596/comments","events_url":"https://api.github.com/repos/fluent/fluentd/issues/596/events","html_url":"https://github.com/fluent/fluentd/issues/596","id":77560727,"number":596,"title":"Fluentd Elasticsearch output stops sending data","user":{"login":"yusufcan141","id":3331781,"avatar_url":"https://avatars.githubusercontent.com/u/3331781?v=3","gravatar_id":"","url":"https://api.github.com/users/yusufcan141","html_url":"https://github.com/yusufcan141","followers_url":"https://api.github.com/users/yusufcan141/followers","following_url":"https://api.github.com/users/yusufcan141/following{/other_user}","gists_url":"https://api.github.com/users/yusufcan141/gists{/gist_id}","starred_url":"https://api.github.com/users/yusufcan141/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yusufcan141/subscriptions","organizations_url":"https://api.github.com/users/yusufcan141/orgs","repos_url":"https://api.github.com/users/yusufcan141/repos","events_url":"https://api.github.com/users/yusufcan141/events{/privacy}","received_events_url":"https://api.github.com/users/yusufcan141/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-05-18T08:31:20Z","updated_at":"2015-05-18T08:45:05Z","closed_at":null,"body":"I'm using elasticsearch output. Every morning at 6.30, all machines stop sending data to elasticsearch and S3. I'm using fluent-plugin-forest also. Here is the error:\r\n\r\n2015-05-14 20:05:14 +0300 [error]: Cannot output messages with tag 'elasticsearch.esData.events_2015_05_01.file.log'\r\n2015-05-14 20:05:14 +0300 [error]: failed to configure/start sub output elasticsearch: can't create Thread: Resource temporarily unavailable\r\n2015-05-14 20:05:14 +0300 [error]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:110:in `initialize'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:110:in `new'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:110:in `start'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:237:in `block in start'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:237:in `each'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:237:in `start'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.8.0/lib/fluent/plugin/out_elasticsearch.rb:44:in `start'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-forest-0.3.0/lib/fluent/plugin/out_forest.rb:133:in `block in plant'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-forest-0.3.0/lib/fluent/plugin/out_forest.rb:128:in `synchronize'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-forest-0.3.0/lib/fluent/plugin/out_forest.rb:128:in `plant'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-forest-0.3.0/lib/fluent/plugin/out_forest.rb:168:in `emit'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:32:in `next'\r\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/buffer.rb:184:in `block in emit'\r\n/opt/td-agent/embedded/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'\r\n"},"comment":{"url":"https://api.github.com/repos/fluent/fluentd/issues/comments/102978622","html_url":"https://github.com/fluent/fluentd/issues/596#issuecomment-102978622","issue_url":"https://api.github.com/repos/fluent/fluentd/issues/596","id":102978622,"user":{"login":"yusufcan141","id":3331781,"avatar_url":"https://avatars.githubusercontent.com/u/3331781?v=3","gravatar_id":"","url":"https://api.github.com/users/yusufcan141","html_url":"https://github.com/yusufcan141","followers_url":"https://api.github.com/users/yusufcan141/followers","following_url":"https://api.github.com/users/yusufcan141/following{/other_user}","gists_url":"https://api.github.com/users/yusufcan141/gists{/gist_id}","starred_url":"https://api.github.com/users/yusufcan141/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yusufcan141/subscriptions","organizations_url":"https://api.github.com/users/yusufcan141/orgs","repos_url":"https://api.github.com/users/yusufcan141/repos","events_url":"https://api.github.com/users/yusufcan141/events{/privacy}","received_events_url":"https://api.github.com/users/yusufcan141/received_events","type":"User","site_admin":false},"created_at":"2015-05-18T08:45:05Z","updated_at":"2015-05-18T08:45:05Z","body":"I have almost 500 tags. But the strange thing is: i'm running fluentd on 3 machines and all of them stops near at 6.30 every day and when i restart td-agent, it continues to send data."}}