{"action":"created","issue":{"url":"https://api.github.com/repos/htgc/fluent-plugin-kafka/issues/15","labels_url":"https://api.github.com/repos/htgc/fluent-plugin-kafka/issues/15/labels{/name}","comments_url":"https://api.github.com/repos/htgc/fluent-plugin-kafka/issues/15/comments","events_url":"https://api.github.com/repos/htgc/fluent-plugin-kafka/issues/15/events","html_url":"https://github.com/htgc/fluent-plugin-kafka/issues/15","id":69512970,"number":15,"title":"\"Failed to send all messages\" exception","user":{"login":"uohzxela","id":4747352,"avatar_url":"https://avatars.githubusercontent.com/u/4747352?v=3","gravatar_id":"","url":"https://api.github.com/users/uohzxela","html_url":"https://github.com/uohzxela","followers_url":"https://api.github.com/users/uohzxela/followers","following_url":"https://api.github.com/users/uohzxela/following{/other_user}","gists_url":"https://api.github.com/users/uohzxela/gists{/gist_id}","starred_url":"https://api.github.com/users/uohzxela/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uohzxela/subscriptions","organizations_url":"https://api.github.com/users/uohzxela/orgs","repos_url":"https://api.github.com/users/uohzxela/repos","events_url":"https://api.github.com/users/uohzxela/events{/privacy}","received_events_url":"https://api.github.com/users/uohzxela/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":19,"created_at":"2015-04-20T05:47:48Z","updated_at":"2015-04-27T03:19:37Z","closed_at":null,"body":"Kafka version: 0.8.1\r\nOS: Ubuntu\r\n\r\nNo idea why fluentd plugin can't send logs to kafka. Kafka is up and running. Tried playing with the timeout and acks attribute in match config but didn't work. Any ideas?\r\n\r\n```\r\n==> hotels_impression_rates_from_tail_log.conf <==\r\n<source>\r\n  type tail\r\n  tag staging.logs.hotels.impressions.rates\r\n  path /var/apps/a/hoteru/shared/log/fluentd/rates.staging.log\r\n  pos_file /var/apps/a/hoteru/shared/log/fluentd/rates.staging.log.pos\r\n  format json\r\n</source>\r\n\r\n==> hotels_impression_rates_to_kafka.conf <==\r\n<match staging.logs.hotels.impressions.rates>\r\n  type kafka\r\n  brokers kafka-staging-a-1.bezurk.org:9092\r\n  default_topic alex\r\n  output_data_type json\r\n</match>\r\n```\r\n\r\n```\r\n2015-04-20 13:23:04 +0800 [info]: brokers has been set directly: [\"kafka-staging-a-1.bezurk.org:9092\"]\r\n2015-04-20 13:23:04 +0800 [info]: adding match pattern=\"td.*.*\" type=\"tdlog\"\r\n2015-04-20 13:23:04 +0800 [info]: adding match pattern=\"debug.**\" type=\"stdout\"\r\n2015-04-20 13:23:04 +0800 [info]: adding source type=\"tail\"\r\n2015-04-20 13:23:04 +0800 [info]: adding source type=\"forward\"\r\n2015-04-20 13:23:04 +0800 [info]: adding source type=\"debug_agent\"\r\n2015-04-20 13:23:04 +0800 [info]: initialized producer kafka\r\n2015-04-20 13:23:04 +0800 [info]: following tail of /var/apps/a/hoteru/shared/log/fluentd/rates.staging.log\r\n2015-04-20 13:23:04 +0800 [info]: listening fluent socket on 0.0.0.0:24224\r\n2015-04-20 13:23:04 +0800 [info]: listening dRuby uri=\"druby://127.0.0.1:24230\" object=\"Engine\"\r\n2015-04-20 13:23:21 +0800 [warn]: Send exception occurred: Failed to send all messages: [#<Poseidon::Message:0x007f97be415fe8 @struct=#<struct Poseidon::Protocol::MessageWithOffsetStruct offset=0, message=#<struct Poseidon::Protocol::MessageStruct magic_type=0, attributes=0, key=nil, value=\"{\\\"search_id\\\":\\\"39790490\\\",\\\"location_id\\\":\\\"6667\\\",\\\"country_code\\\":\\\"SG\\\",\\\"location_name\\\":\\\"Singapore\\\",\\\"locale\\\":\\\"en\\\",\\\"site_code\\\":\\\"US\\\",\\\"wego_visitor_id\\\":\\\"1429252550160-QmuHwg\\\",\\\"wego_session_id\\\":\\\"1429506752934-lJt96p\\\",\\\"device_type\\\":\\\"desktop_linux\\\",\\\"ip\\\":\\\"219.74.18.84\\\",\\\"timestamp\\\":\\\"2015-04-20T13:23:14.849+08:00\\\",\\\"hotel_name\\\":\\\"Marina Bay Sands\\\",\\\"hotel_id\\\":\\\"258101\\\",\\\"ecpc_max\\\":0.8,\\\"ecpc_min\\\":0.05,\\\"ecpc_card_min\\\":0.7,\\\"hotel_rank\\\":1,\\\"ecpc\\\":0.8,\\\"provider_hotel_id\\\":\\\"245881\\\",\\\"provider_code\\\":\\\"booking.com\\\",\\\"price_usd\\\":394.1027,\\\"description\\\":\\\"Premier Double or Twin Room - Free cancellation - Book now, pay when you stay - Free WiFi - Found your room online at a lower price? We'll match it.\\\",\\\"rate_rank\\\":1,\\\"ecpc_rank\\\":1,\\\"bucket\\\":\\\"0.8\\\",\\\"placement\\\":\\\"listing_sponsored\\\",\\\"params\\\":{}}\">>, @topic=\"alex\">] remaining\r\n2015-04-20 13:23:21 +0800 [info]: initialized producer kafka\r\n2015-04-20 13:23:21 +0800 [warn]: emit transaction failed: error_class=RuntimeError error=\"Failed to send all messages: [#<Poseidon::Message:0x007f97be415fe8 @struct=#<struct Poseidon::Protocol::MessageWithOffsetStruct offset=0, message=#<struct Poseidon::Protocol::MessageStruct magic_type=0, attributes=0, key=nil, value=\\\"{\\\\\\\"search_id\\\\\\\":\\\\\\\"39790490\\\\\\\",\\\\\\\"location_id\\\\\\\":\\\\\\\"6667\\\\\\\",\\\\\\\"country_code\\\\\\\":\\\\\\\"SG\\\\\\\",\\\\\\\"location_name\\\\\\\":\\\\\\\"Singapore\\\\\\\",\\\\\\\"locale\\\\\\\":\\\\\\\"en\\\\\\\",\\\\\\\"site_code\\\\\\\":\\\\\\\"US\\\\\\\",\\\\\\\"wego_visitor_id\\\\\\\":\\\\\\\"1429252550160-QmuHwg\\\\\\\",\\\\\\\"wego_session_id\\\\\\\":\\\\\\\"1429506752934-lJt96p\\\\\\\",\\\\\\\"device_type\\\\\\\":\\\\\\\"desktop_linux\\\\\\\",\\\\\\\"ip\\\\\\\":\\\\\\\"219.74.18.84\\\\\\\",\\\\\\\"timestamp\\\\\\\":\\\\\\\"2015-04-20T13:23:14.849+08:00\\\\\\\",\\\\\\\"hotel_name\\\\\\\":\\\\\\\"Marina Bay Sands\\\\\\\",\\\\\\\"hotel_id\\\\\\\":\\\\\\\"258101\\\\\\\",\\\\\\\"ecpc_max\\\\\\\":0.8,\\\\\\\"ecpc_min\\\\\\\":0.05,\\\\\\\"ecpc_card_min\\\\\\\":0.7,\\\\\\\"hotel_rank\\\\\\\":1,\\\\\\\"ecpc\\\\\\\":0.8,\\\\\\\"provider_hotel_id\\\\\\\":\\\\\\\"245881\\\\\\\",\\\\\\\"provider_code\\\\\\\":\\\\\\\"booking.com\\\\\\\",\\\\\\\"price_usd\\\\\\\":394.1027,\\\\\\\"description\\\\\\\":\\\\\\\"Premier Double or Twin Room - Free cancellation - Book now, pay when you stay - Free WiFi - Found your room online at a lower price? We'll match it.\\\\\\\",\\\\\\\"rate_rank\\\\\\\":1,\\\\\\\"ecpc_rank\\\\\\\":1,\\\\\\\"bucket\\\\\\\":\\\\\\\"0.8\\\\\\\",\\\\\\\"placement\\\\\\\":\\\\\\\"listing_sponsored\\\\\\\",\\\\\\\"params\\\\\\\":{}}\\\">>, @topic=\\\"alex\\\">] remaining\" tag=\"staging.logs.hotels.impressions.rates\"\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/poseidon-0.0.5/lib/poseidon/sync_producer.rb:67:in `send_messages'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/poseidon-0.0.5/lib/poseidon/producer.rb:163:in `send_messages'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-kafka-0.0.12/lib/fluent/plugin/out_kafka.rb:135:in `block in emit'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event.rb:128:in `call'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event.rb:128:in `block in each'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event.rb:127:in `each'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event.rb:127:in `each'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-kafka-0.0.12/lib/fluent/plugin/out_kafka.rb:128:in `emit'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event_router.rb:88:in `emit_stream'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:229:in `receive_lines'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:320:in `call'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:320:in `wrap_receive_lines'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:513:in `call'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:513:in `on_notify'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:345:in `on_notify'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:446:in `call'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:446:in `on_change'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/cool.io-1.3.0/lib/cool.io/loop.rb:88:in `run_once'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/cool.io-1.3.0/lib/cool.io/loop.rb:88:in `run'\r\n  2015-04-20 13:23:21 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:214:in `run'\r\n```"},"comment":{"url":"https://api.github.com/repos/htgc/fluent-plugin-kafka/issues/comments/96486020","html_url":"https://github.com/htgc/fluent-plugin-kafka/issues/15#issuecomment-96486020","issue_url":"https://api.github.com/repos/htgc/fluent-plugin-kafka/issues/15","id":96486020,"user":{"login":"uohzxela","id":4747352,"avatar_url":"https://avatars.githubusercontent.com/u/4747352?v=3","gravatar_id":"","url":"https://api.github.com/users/uohzxela","html_url":"https://github.com/uohzxela","followers_url":"https://api.github.com/users/uohzxela/followers","following_url":"https://api.github.com/users/uohzxela/following{/other_user}","gists_url":"https://api.github.com/users/uohzxela/gists{/gist_id}","starred_url":"https://api.github.com/users/uohzxela/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uohzxela/subscriptions","organizations_url":"https://api.github.com/users/uohzxela/orgs","repos_url":"https://api.github.com/users/uohzxela/repos","events_url":"https://api.github.com/users/uohzxela/events{/privacy}","received_events_url":"https://api.github.com/users/uohzxela/received_events","type":"User","site_admin":false},"created_at":"2015-04-27T03:19:37Z","updated_at":"2015-04-27T03:19:37Z","body":"> Perhaps, you should setup hostname in your server.properties file on broker host.\r\n> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whycan%27tmyconsumers/producersconnecttothebrokers?\r\n\r\nThanks for the help! Tried setting the host.name, and it can detect the remote kafka host but got a new error instead. Any ideas?\r\n\r\n```\r\n2015-04-27 11:00:59 +0800 [info]: adding match pattern=\"td.*.*\" type=\"tdlog\"\r\n2015-04-27 11:00:59 +0800 [info]: adding match pattern=\"debug.**\" type=\"stdout\"\r\n2015-04-27 11:00:59 +0800 [info]: adding source type=\"tail\"\r\n2015-04-27 11:00:59 +0800 [info]: adding source type=\"forward\"\r\n2015-04-27 11:00:59 +0800 [info]: adding source type=\"debug_agent\"\r\n2015-04-27 11:00:59 +0800 [info]: brokers has been refreshed via Zookeeper: [\"kafka-staging-a-1.bezurk.org:9092\"]\r\n2015-04-27 11:00:59 +0800 [info]: initialized producer kafka\r\n2015-04-27 11:00:59 +0800 [info]: following tail of /var/apps/a/hoteru/shared/log/fluentd/rates.staging.log\r\n2015-04-27 11:00:59 +0800 [info]: listening fluent socket on 0.0.0.0:24224\r\n2015-04-27 11:00:59 +0800 [info]: listening dRuby uri=\"druby://127.0.0.1:24230\" object=\"Engine\"\r\n2015-04-27 11:01:06 +0800 [warn]: Send exception occurred: Poseidon::Errors::UnableToFetchMetadata\r\n2015-04-27 11:01:06 +0800 [info]: brokers has been refreshed via Zookeeper: [\"kafka-staging-a-1.bezurk.org:9092\"]\r\n2015-04-27 11:01:06 +0800 [info]: initialized producer kafka\r\n2015-04-27 11:01:06 +0800 [warn]: emit transaction failed: error_class=Poseidon::Errors::UnableToFetchMetadata error=\"Poseidon::Errors::UnableToFetchMetadata\" tag=\"staging.logs.hotels.impressions.rates\"\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/poseidon-0.0.5/lib/poseidon/broker_pool.rb:33:in `fetch_metadata'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/poseidon-0.0.5/lib/poseidon/sync_producer.rb:134:in `refresh_metadata'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/poseidon-0.0.5/lib/poseidon/sync_producer.rb:46:in `send_messages'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/poseidon-0.0.5/lib/poseidon/producer.rb:163:in `send_messages'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-kafka-0.0.12/lib/fluent/plugin/out_kafka.rb:135:in `block in emit'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event.rb:128:in `call'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event.rb:128:in `block in each'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event.rb:127:in `each'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event.rb:127:in `each'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-kafka-0.0.12/lib/fluent/plugin/out_kafka.rb:128:in `emit'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/event_router.rb:88:in `emit_stream'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:229:in `receive_lines'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:320:in `call'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:320:in `wrap_receive_lines'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:513:in `call'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:513:in `on_notify'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:345:in `on_notify'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:446:in `call'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:446:in `on_change'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/cool.io-1.3.0/lib/cool.io/loop.rb:88:in `run_once'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/cool.io-1.3.0/lib/cool.io/loop.rb:88:in `run'\r\n  2015-04-27 11:01:06 +0800 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/in_tail.rb:214:in `run'\r\n```"}}