{"action":"closed","issue":{"url":"https://api.github.com/repos/alibaba/RocketMQ/issues/28","labels_url":"https://api.github.com/repos/alibaba/RocketMQ/issues/28/labels{/name}","comments_url":"https://api.github.com/repos/alibaba/RocketMQ/issues/28/comments","events_url":"https://api.github.com/repos/alibaba/RocketMQ/issues/28/events","html_url":"https://github.com/alibaba/RocketMQ/issues/28","id":52083474,"number":28,"title":"重启Broker-Master 会导致之前已成功消费过的消息被全部重新消费一次！","user":{"login":"flysqrlboy","id":7646548,"avatar_url":"https://avatars.githubusercontent.com/u/7646548?v=3","gravatar_id":"","url":"https://api.github.com/users/flysqrlboy","html_url":"https://github.com/flysqrlboy","followers_url":"https://api.github.com/users/flysqrlboy/followers","following_url":"https://api.github.com/users/flysqrlboy/following{/other_user}","gists_url":"https://api.github.com/users/flysqrlboy/gists{/gist_id}","starred_url":"https://api.github.com/users/flysqrlboy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flysqrlboy/subscriptions","organizations_url":"https://api.github.com/users/flysqrlboy/orgs","repos_url":"https://api.github.com/users/flysqrlboy/repos","events_url":"https://api.github.com/users/flysqrlboy/events{/privacy}","received_events_url":"https://api.github.com/users/flysqrlboy/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2014-12-16T07:59:45Z","updated_at":"2015-03-24T03:45:39Z","closed_at":"2015-03-24T03:45:39Z","body":"重启Broker-Master 会导致之前已成功消费过的消息被全部重新消费一次！\r\n\r\n1。【环境】\r\nrocketmq version : 3.1.9 \r\n集群方式： 2个nameserver；1个broker-master ，1个broker-slave，异步复制\r\n\r\n2。【过程重现】\r\n（1）关闭broker-master 之前，topic=mobileSyncDataTopic 的消费进度：已成功消费50条消息。\r\n\r\nmqadmin  consumerProgress -g mobileSyncDataConsumerGroup -n 10.10.10.131:9876\r\n#Topic                            #Broker Name                      #QID  #Broker Offset        #Consumer Offset      #Diff\r\n%RETRY%mobileSyncData       broker-a                          0     0                     0                     0\r\nmobileSyncDataTopic               broker-a                          0     13                    13                    0\r\nmobileSyncDataTopic               broker-a                          1     13                    13                    0\r\nmobileSyncDataTopic               broker-a                          2     12                    12                    0\r\nmobileSyncDataTopic               broker-a                          3     12                    12                    0\r\n\r\n（2）关闭broker-master 后，offset重置为0，Diff也为0\r\n\r\nmqadmin  consumerProgress -g mobileSyncDataConsumerGroup -n 10.10.10.131:9876\r\n#Topic                            #Broker Name                      #QID  #Broker Offset        #Consumer Offset      #Diff\r\n%RETRY%mobileSync              broker-a                          0     0                     0                     0\r\nmobileSyncDataTopic               broker-a                          0     0                     0                     0\r\nmobileSyncDataTopic               broker-a                          1     0                     0                     0\r\nmobileSyncDataTopic               broker-a                          2     0                     0                     0\r\nmobileSyncDataTopic               broker-a                          3     0                     0                     0\r\n\r\n（3）重启broker-master后，offset 、diff 会恢复成重启前的值。且已经在重启前成功消费过的50条消息会重新消费一次。\r\n \r\nmqadmin  consumerProgress -g mobileSyncDataConsumerGroup -n 10.10.10.131:9876 \r\n#Topic                            #Broker Name                      #QID  #Broker Offset        #Consumer Offset      #Diff\r\n%RETRY%mobileSync              broker-a                          0     0                     0                     0\r\nmobileSyncDataTopic               broker-a                          0     13                    13                    0\r\nmobileSyncDataTopic               broker-a                          1     13                    13                    0\r\nmobileSyncDataTopic               broker-a                          2     12                    12                    0\r\nmobileSyncDataTopic               broker-a                          3     12                    12                    0\r\n\r\n下面分别是重启前的消费日志和重启后的消费日志，可以看到，消息确实被重新消费了【日志太多，节选三条消息对比如下】！\r\n\r\n【重启前消费日志】\r\nConsumeMessageThread_1 Receive New Messages: [MessageExt [queueId=0, storeSize=146, queueOffset=0, sysFlag=0, bornTimestamp=1418712824540, bornHost=/10.10.10.1:55629, storeTimestamp=1418712824677, storeHost=/10.10.10.131:10911, msgId=0A0A0A8300002A9F000000000089920F, commitLogOffset=9015823, bodyCRC=613185359, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message [topic=mobileSyncDataTopic, flag=0, properties={TAGS=TagA, WAIT=true, MAX_OFFSET=13, MIN_OFFSET=0}, body=16]]]\r\nConsumeMessageThread_2 Receive New Messages: [MessageExt [queueId=0, storeSize=146, queueOffset=1, sysFlag=0, bornTimestamp=1418712825062, bornHost=/10.10.10.1:55629, storeTimestamp=1418712824759, storeHost=/10.10.10.131:10911, msgId=0A0A0A8300002A9F0000000000899457, commitLogOffset=9016407, bodyCRC=601994070, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message [topic=mobileSyncDataTopic, flag=0, properties={TAGS=TagA, WAIT=true, MAX_OFFSET=13, MIN_OFFSET=0}, body=16]]]\r\nConsumeMessageThread_5 Receive New Messages: [MessageExt [queueId=0, storeSize=147, queueOffset=4, sysFlag=0, bornTimestamp=1418712825175, bornHost=/10.10.10.1:55629, storeTimestamp=1418712824884, storeHost=/10.10.10.131:10911, msgId=0A0A0A8300002A9F0000000000899B35, commitLogOffset=9018165, bodyCRC=1659149091, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message [topic=mobileSyncDataTopic, flag=0, properties={TAGS=TagA, WAIT=true, MAX_OFFSET=13, MIN_OFFSET=0}, body=17]]]\r\n\r\n\r\n\r\n【重启后消费日志】\r\nConsumeMessageThread_19 Receive New Messages: [MessageExt [queueId=0, storeSize=146, queueOffset=0, sysFlag=0, bornTimestamp=1418712824540, bornHost=/10.10.10.1:55629, storeTimestamp=1418712824677, storeHost=/10.10.10.131:10911, msgId=0A0A0A8300002A9F000000000089920F, commitLogOffset=9015823, bodyCRC=613185359, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message [topic=mobileSyncDataTopic, flag=0, properties={TAGS=TagA, WAIT=true, MAX_OFFSET=13, MIN_OFFSET=0}, body=16]]]\r\nConsumeMessageThread_19 Receive New Messages: [MessageExt [queueId=0, storeSize=146, queueOffset=1, sysFlag=0, bornTimestamp=1418712825062, bornHost=/10.10.10.1:55629, storeTimestamp=1418712824759, storeHost=/10.10.10.131:10911, msgId=0A0A0A8300002A9F0000000000899457, commitLogOffset=9016407, bodyCRC=601994070, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message [topic=mobileSyncDataTopic, flag=0, properties={TAGS=TagA, WAIT=true, MAX_OFFSET=13, MIN_OFFSET=0}, body=16]]]\r\nConsumeMessageThread_18 Receive New Messages: [MessageExt [queueId=0, storeSize=147, queueOffset=4, sysFlag=0, bornTimestamp=1418712825175, bornHost=/10.10.10.1:55629, storeTimestamp=1418712824884, storeHost=/10.10.10.131:10911, msgId=0A0A0A8300002A9F0000000000899B35, commitLogOffset=9018165, bodyCRC=1659149091, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message [topic=mobileSyncDataTopic, flag=0, properties={TAGS=TagA, WAIT=true, MAX_OFFSET=13, MIN_OFFSET=0}, body=17]]]\r\n\r\n\r\n3.【问题】\r\n（1）如何避免在broker-master重启后以前已被消费过的消息再一次被全部消费？\r\n（2）当broker-master宕机后，broker-slave不会自动切换为master。在broker-master宕掉后，再发消息会报错：\r\n          No route info of this topic, mobileSyncDataTopic\r\n          See https://github.com/alibaba/RocketMQ/issues/264 for further details.\r\n     at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.sendDefaultImpl( DefaultMQProducerImpl.java:624)\r\n     at com.alibaba.rocketmq.client.impl.producer.DefaultMQProducerImpl.send( DefaultMQProducerImpl.java:1050)\r\n     at com.alibaba.rocketmq.client.producer.DefaultMQProducer.send( DefaultMQProducer.java:122)\r\n          也就是说master-slave没有起到HA failover的效果。\r\n"}}