Kafka consumer Failed to find leader for Set and Fetching topic metadata with correlation id 4 for topics failed
<p>So I'm getting this error with a single kafka 0.8.2.0 broker and a java consumer running in mesos. I have to say it used to work, and AFAIK nothing has changed. </p>

<p>Here's the error when the consumer runs in mesos:</p>

<pre><code>16/03/17 05:44:23 WARN ConsumerFetcherManager$LeaderFinderThread: [alertConsumer_secasprddb01-1-1458193432645-387abff0-leader-finder-thread], Failed to find leader for Set([events,1], [events,14], [events,2], [events,8], [events,7], [events,10], [events,3], [events,11], [events,5], [events,0], [events,12], [events,13], [events,6], [events,4], [events,9]) kafka.common.KafkaException: fetching topic metadata for topics [Set(events)] from broker [ArrayBuffer(id:0,host:prodMessage01-1.blah.com,port:9092)] failed at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) Caused by: java.net.SocketTimeoutException
</code></pre>

<p>I even went as far as completely deleting the topic and recreating it form scratch and still the same error. </p>

<p>If I run the script kafka-console-consumer.sh on the same box as the broker, I'm still getting the error:</p>

<pre><code>Failed to find leader for Set([events,1], [events,14], [events,2], [events,8], [events,7], [events,10], [events,3], [events,11], [events,5], [events,0], [events,12], [events,13], [events,6], [events,9], [events,4]) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread) kafka.common.KafkaException: fetching topic metadata for topics [Set(events)] from broker [ArrayBuffer(id:0,host:prodMessage01-1.blah.com,port:9092)] failed
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
    at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:93)
    at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:60) Caused by: java.net.SocketTimeoutException
</code></pre>

<p>So even when everyting is local to the broker, it still is failing. How is this even possible?</p>

<p>I don't see how I can get past this if I've actually deleted and recreated the topic. </p>

<p>If I do a topic describe (kafka-topics.sh -- describe), it works fine:
<code>
Topic:events    PartitionCount:15       ReplicationFactor:1     Configs:
        Topic: events   Partition: 0    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 1    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 2    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 3    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 4    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 5    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 6    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 7    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 8    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 9    Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 10   Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 11   Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 12   Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 13   Leader: 0       Replicas: 0     Isr: 0
        Topic: events   Partition: 14   Leader: 0       Replicas: 0     Isr: 0
</code></p>