Kafka new consumer: (re)set and commit offsets with using assign, not subscribe
<p>Using the new Kafka Java consumer api, I run a single consumer to consume messages. When all available messages are consumed, I kill it with <code>kill -15</code>.</p>

<p>Now I would like to reset the offsets to start. I would like to avoid to just use a different consumer group. What I tried is the following sequence of calls, using the same group as the consumer that just had finished reading the data.</p>

<pre><code>assign(topicPartition);
OffsetAndMetadata om = new OffsetAndMetadata(0);
commitSync(Collections.singletonMap(topicPartition, 0));
</code></pre>

<p>I thought I had got this working in a test, but now I always just get:</p>

<pre><code>ERROR internals.ConsumerCoordinator: Error UNKNOWN_MEMBER_ID occurred while committing offsets for group queue
Exception in thread "main" org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed due to group rebalance
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:552)
</code></pre>

<p>Is it in principle wrong to combine <code>assign</code> with <code>commitSync</code>, possibly because only <code>subscribe</code> and <code>commitSync</code> go together? The docs only say that <code>assign</code> does not go along with <code>subscribe</code>, but I thought this applies only in one consumer process. (In fact I was even hoping to run the offset-reset consumer while the other consumer is up, hoping that the other one might notice the offset change and start over again. But shutting it down first is fine too.)</p>

<p>Any ideas?</p>