{"action":"created","issue":{"url":"https://api.github.com/repos/elasticsearch/logstash/issues/2212","labels_url":"https://api.github.com/repos/elasticsearch/logstash/issues/2212/labels{/name}","comments_url":"https://api.github.com/repos/elasticsearch/logstash/issues/2212/comments","events_url":"https://api.github.com/repos/elasticsearch/logstash/issues/2212/events","html_url":"https://github.com/elasticsearch/logstash/issues/2212","id":51437248,"number":2212,"title":"RabbitMQ queues get deleted no matter what on shutdown","user":{"login":"teu","id":589133,"avatar_url":"https://avatars.githubusercontent.com/u/589133?v=3","gravatar_id":"","url":"https://api.github.com/users/teu","html_url":"https://github.com/teu","followers_url":"https://api.github.com/users/teu/followers","following_url":"https://api.github.com/users/teu/following{/other_user}","gists_url":"https://api.github.com/users/teu/gists{/gist_id}","starred_url":"https://api.github.com/users/teu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/teu/subscriptions","organizations_url":"https://api.github.com/users/teu/orgs","repos_url":"https://api.github.com/users/teu/repos","events_url":"https://api.github.com/users/teu/events{/privacy}","received_events_url":"https://api.github.com/users/teu/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elasticsearch/logstash/labels/1.5","name":"1.5","color":"ededed"},{"url":"https://api.github.com/repos/elasticsearch/logstash/labels/needs+more+info","name":"needs more info","color":"f7c6c7"}],"state":"closed","locked":false,"assignee":{"login":"jsvd","id":31809,"avatar_url":"https://avatars.githubusercontent.com/u/31809?v=3","gravatar_id":"","url":"https://api.github.com/users/jsvd","html_url":"https://github.com/jsvd","followers_url":"https://api.github.com/users/jsvd/followers","following_url":"https://api.github.com/users/jsvd/following{/other_user}","gists_url":"https://api.github.com/users/jsvd/gists{/gist_id}","starred_url":"https://api.github.com/users/jsvd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsvd/subscriptions","organizations_url":"https://api.github.com/users/jsvd/orgs","repos_url":"https://api.github.com/users/jsvd/repos","events_url":"https://api.github.com/users/jsvd/events{/privacy}","received_events_url":"https://api.github.com/users/jsvd/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/elasticsearch/logstash/milestones/2","labels_url":"https://api.github.com/repos/elasticsearch/logstash/milestones/2/labels","id":624113,"number":2,"title":"v1.5.0","description":"","creator":{"login":"colinsurprenant","id":2010,"avatar_url":"https://avatars.githubusercontent.com/u/2010?v=3","gravatar_id":"","url":"https://api.github.com/users/colinsurprenant","html_url":"https://github.com/colinsurprenant","followers_url":"https://api.github.com/users/colinsurprenant/followers","following_url":"https://api.github.com/users/colinsurprenant/following{/other_user}","gists_url":"https://api.github.com/users/colinsurprenant/gists{/gist_id}","starred_url":"https://api.github.com/users/colinsurprenant/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/colinsurprenant/subscriptions","organizations_url":"https://api.github.com/users/colinsurprenant/orgs","repos_url":"https://api.github.com/users/colinsurprenant/repos","events_url":"https://api.github.com/users/colinsurprenant/events{/privacy}","received_events_url":"https://api.github.com/users/colinsurprenant/received_events","type":"User","site_admin":false},"open_issues":26,"closed_issues":200,"state":"open","created_at":"2014-04-08T16:53:38Z","updated_at":"2015-01-15T23:25:33Z","due_on":null,"closed_at":null},"comments":8,"created_at":"2014-12-09T15:02:19Z","updated_at":"2015-01-15T23:25:33Z","closed_at":"2015-01-15T23:25:33Z","body":"Hi,\r\n\r\nas much as we love logstash+elasticsearch, shipping with logstash-forwarder->logstash has proven to be quite unreliable (logstash getting overwhelmed quickly and some race condition bounding connections = logs lost), today we tested moving messages though beaver->rabbitmq->logstash. \r\n\r\nThe latter approach in theory should let us to process any number of logs (horizontally scaling on AWS) and remove SPoF's. Idea is to have beavers ship to RabbitMQ cluster, another dynamic logstash cluster would consume the messages and ship to yet another elasticsearch cluster. When any one thing gets to small we can always adjust it by simply adding machines.\r\n\r\nThe problem we found is that the logs from the rabbitmq input vanish into thin air. \r\n\r\nError message that we get when we kill logstash (perhaps not relevant) \r\n```\r\nNoMethodError: undefined method `cancel' for nil:NilClass\r\n  shutdown_consumer at /home/ubuntu/logstash-1.4.2/lib/logstash/inputs/rabbitmq/march_hare.rb:126\r\n           teardown at /home/ubuntu/logstash-1.4.2/lib/logstash/inputs/rabbitmq/march_hare.rb:68\r\n           shutdown at /home/ubuntu/logstash-1.4.2/lib/logstash/pipeline.rb:247\r\n               each at org/jruby/RubyArray.java:1613\r\n           shutdown at /home/ubuntu/logstash-1.4.2/lib/logstash/pipeline.rb:246\r\n               each at org/jruby/RubyArray.java:1613\r\n           shutdown at /home/ubuntu/logstash-1.4.2/lib/logstash/pipeline.rb:234\r\n            execute at /home/ubuntu/logstash-1.4.2/lib/logstash/agent.rb:120\r\n               call at org/jruby/RubyProc.java:271\r\n               each at org/jruby/RubyArray.java:1613\r\n    simulate_signal at /home/ubuntu/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/stud-0.0.17/lib/stud/trap.rb:46\r\n               trap at /home/ubuntu/logstash-1.4.2/vendor/bundle/jruby/1.9/gems/stud-0.0.17/lib/stud/trap.rb:26\r\n               call at org/jruby/RubyProc.java:271\r\n\r\n```\r\n\r\nConfiguration is following:\r\n```\r\nrabbitmq {\r\n\thost => [moderated]\r\n\tack => true\r\n\tauto_delete => false\r\n\tcodec => \"json\"\r\n\ttype => \"http-access-log\"\r\n\tkey => \"logstash\"\r\n\tpassive => true\r\n\tprefetch_count => 10\r\n\tqueue => \"beaver-access-logs\"\r\n\tuser => [moderated]\r\n\tpassword => [moderated]\r\n}\r\n\r\n```\r\n\r\nAnother problem is, that the messages have been set to ACK but seam to be vanishing into thin air (not a word in the logs) but getting ACKed before being shipped of anywhere. The same logstash agent is working with several lumberjack inputs that are moving logs to elasticsearch just fine and the format is the same as I am testing on a machine that normally has logstash-forwarder running on it. On a side note logstash is restarting a lot without any log entry or apparent cause.\r\n\r\n Please advise."},"comment":{"url":"https://api.github.com/repos/elasticsearch/logstash/issues/comments/70182370","html_url":"https://github.com/elasticsearch/logstash/issues/2212#issuecomment-70182370","issue_url":"https://api.github.com/repos/elasticsearch/logstash/issues/2212","id":70182370,"user":{"login":"suyograo","id":1595958,"avatar_url":"https://avatars.githubusercontent.com/u/1595958?v=3","gravatar_id":"","url":"https://api.github.com/users/suyograo","html_url":"https://github.com/suyograo","followers_url":"https://api.github.com/users/suyograo/followers","following_url":"https://api.github.com/users/suyograo/following{/other_user}","gists_url":"https://api.github.com/users/suyograo/gists{/gist_id}","starred_url":"https://api.github.com/users/suyograo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/suyograo/subscriptions","organizations_url":"https://api.github.com/users/suyograo/orgs","repos_url":"https://api.github.com/users/suyograo/repos","events_url":"https://api.github.com/users/suyograo/events{/privacy}","received_events_url":"https://api.github.com/users/suyograo/received_events","type":"User","site_admin":false},"created_at":"2015-01-15T23:25:32Z","updated_at":"2015-01-15T23:25:32Z","body":"@teu Closing this, please reopen if this is not fixed yet"}}