{"action":"created","issue":{"url":"https://api.github.com/repos/Netflix/dynomite/issues/182","labels_url":"https://api.github.com/repos/Netflix/dynomite/issues/182/labels{/name}","comments_url":"https://api.github.com/repos/Netflix/dynomite/issues/182/comments","events_url":"https://api.github.com/repos/Netflix/dynomite/issues/182/events","html_url":"https://github.com/Netflix/dynomite/issues/182","id":112328064,"number":182,"title":"dynomite crash after the peer node down when gossip is working","user":{"login":"wuyingfengsui","id":1544491,"avatar_url":"https://avatars.githubusercontent.com/u/1544491?v=3","gravatar_id":"","url":"https://api.github.com/users/wuyingfengsui","html_url":"https://github.com/wuyingfengsui","followers_url":"https://api.github.com/users/wuyingfengsui/followers","following_url":"https://api.github.com/users/wuyingfengsui/following{/other_user}","gists_url":"https://api.github.com/users/wuyingfengsui/gists{/gist_id}","starred_url":"https://api.github.com/users/wuyingfengsui/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wuyingfengsui/subscriptions","organizations_url":"https://api.github.com/users/wuyingfengsui/orgs","repos_url":"https://api.github.com/users/wuyingfengsui/repos","events_url":"https://api.github.com/users/wuyingfengsui/events{/privacy}","received_events_url":"https://api.github.com/users/wuyingfengsui/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"shailesh33","id":12148455,"avatar_url":"https://avatars.githubusercontent.com/u/12148455?v=3","gravatar_id":"","url":"https://api.github.com/users/shailesh33","html_url":"https://github.com/shailesh33","followers_url":"https://api.github.com/users/shailesh33/followers","following_url":"https://api.github.com/users/shailesh33/following{/other_user}","gists_url":"https://api.github.com/users/shailesh33/gists{/gist_id}","starred_url":"https://api.github.com/users/shailesh33/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shailesh33/subscriptions","organizations_url":"https://api.github.com/users/shailesh33/orgs","repos_url":"https://api.github.com/users/shailesh33/repos","events_url":"https://api.github.com/users/shailesh33/events{/privacy}","received_events_url":"https://api.github.com/users/shailesh33/received_events","type":"User","site_admin":false},"milestone":null,"comments":8,"created_at":"2015-10-20T09:51:14Z","updated_at":"2015-10-27T06:42:15Z","closed_at":null,"body":"I start two nodes with gossip. In the begining, everything is ok. But if I shut down one of them, another will crash in the few seconds:\r\n```\r\n[2015-10-20 17:31:4.191] core_close_log:290 close PEER_SERVER 10 '192.168.2.196:8101' on event FFFFFF eof 0 done 0 rb 0 sb 0: Connection refused\r\n[2015-10-20 17:31:4.191] dnode_peer_close:572 dyn: dnode_peer_close on peer '192.168.2.196:8101:rack1:dc1:437425602'\r\n[2015-10-20 17:31:4.191] stats_pool_to_metric:1502 metric 'peer_err' in pool 0\r\n[2015-10-20 17:31:4.191] _stats_pool_incr:1521 incr field 'peer_err' to 1\r\n[2015-10-20 17:31:4.191] dnode_req_peer_dequeue_imsgq:56 conn 0x121ded0 dequeue inq 92:0\r\n[2015-10-20 17:31:4.191] stats_pool_to_metric:1502 metric 'peer_in_queue' in pool 0\r\n[2015-10-20 17:31:4.191] _stats_pool_decr:1539 decr field 'peer_in_queue' to 0\r\n[2015-10-20 17:31:4.191] stats_pool_to_metric:1502 metric 'peer_in_queue_bytes' in pool 0\r\n[2015-10-20 17:31:4.191] _stats_pool_decr_by:1569 decr by field 'peer_in_queue_bytes' to 0\r\n[2015-10-20 17:31:4.191] dnode_peer_ack_err:538 Assertion Failed: conn:PEER_SERVER c_conn:PEER_SERVER, req 92:0\r\n[2015-10-20 17:31:4.191] dn_assert:330 assert 'c_conn->type == CONN_CLIENT' failed @ (dyn_dnode_peer.c, 538)\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [0] src/dynomite() [0x41788a]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [1] src/dynomite(dnode_peer_close+0x1cc) [0x417c66]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [2] src/dynomite() [0x40ead3]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [3] src/dynomite() [0x40eb84]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [4] src/dynomite(core_core+0xf3) [0x40ee64]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [5] src/dynomite(event_wait+0x1b9) [0x4519a3]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [6] src/dynomite(core_loop+0x60) [0x40f6eb]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [7] src/dynomite() [0x441d70]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [8] src/dynomite(main+0x137) [0x441ef0]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [9] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fccb292476d]\r\n[2015-10-20 17:31:4.192] dn_stacktrace:320 [10] src/dynomite() [0x40c8f5]\r\n```"},"comment":{"url":"https://api.github.com/repos/Netflix/dynomite/issues/comments/151393985","html_url":"https://github.com/Netflix/dynomite/issues/182#issuecomment-151393985","issue_url":"https://api.github.com/repos/Netflix/dynomite/issues/182","id":151393985,"user":{"login":"wuyingfengsui","id":1544491,"avatar_url":"https://avatars.githubusercontent.com/u/1544491?v=3","gravatar_id":"","url":"https://api.github.com/users/wuyingfengsui","html_url":"https://github.com/wuyingfengsui","followers_url":"https://api.github.com/users/wuyingfengsui/followers","following_url":"https://api.github.com/users/wuyingfengsui/following{/other_user}","gists_url":"https://api.github.com/users/wuyingfengsui/gists{/gist_id}","starred_url":"https://api.github.com/users/wuyingfengsui/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wuyingfengsui/subscriptions","organizations_url":"https://api.github.com/users/wuyingfengsui/orgs","repos_url":"https://api.github.com/users/wuyingfengsui/repos","events_url":"https://api.github.com/users/wuyingfengsui/events{/privacy}","received_events_url":"https://api.github.com/users/wuyingfengsui/received_events","type":"User","site_admin":false},"created_at":"2015-10-27T06:42:15Z","updated_at":"2015-10-27T06:42:15Z","body":"I think it doesn't have to ack error on this condition. Because the peer is shutdown."}}