{"action":"created","issue":{"url":"https://api.github.com/repos/atomix/copycat/issues/20","labels_url":"https://api.github.com/repos/atomix/copycat/issues/20/labels{/name}","comments_url":"https://api.github.com/repos/atomix/copycat/issues/20/comments","events_url":"https://api.github.com/repos/atomix/copycat/issues/20/events","html_url":"https://github.com/atomix/copycat/issues/20","id":112246932,"number":20,"title":"Support for a Leader to leave the cluster cleanly","user":{"login":"madjam","id":8388825,"avatar_url":"https://avatars.githubusercontent.com/u/8388825?v=3","gravatar_id":"","url":"https://api.github.com/users/madjam","html_url":"https://github.com/madjam","followers_url":"https://api.github.com/users/madjam/followers","following_url":"https://api.github.com/users/madjam/following{/other_user}","gists_url":"https://api.github.com/users/madjam/gists{/gist_id}","starred_url":"https://api.github.com/users/madjam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/madjam/subscriptions","organizations_url":"https://api.github.com/users/madjam/orgs","repos_url":"https://api.github.com/users/madjam/repos","events_url":"https://api.github.com/users/madjam/events{/privacy}","received_events_url":"https://api.github.com/users/madjam/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/atomix/copycat/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"kuujo","id":823318,"avatar_url":"https://avatars.githubusercontent.com/u/823318?v=3","gravatar_id":"","url":"https://api.github.com/users/kuujo","html_url":"https://github.com/kuujo","followers_url":"https://api.github.com/users/kuujo/followers","following_url":"https://api.github.com/users/kuujo/following{/other_user}","gists_url":"https://api.github.com/users/kuujo/gists{/gist_id}","starred_url":"https://api.github.com/users/kuujo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kuujo/subscriptions","organizations_url":"https://api.github.com/users/kuujo/orgs","repos_url":"https://api.github.com/users/kuujo/repos","events_url":"https://api.github.com/users/kuujo/events{/privacy}","received_events_url":"https://api.github.com/users/kuujo/received_events","type":"User","site_admin":false},"milestone":null,"comments":3,"created_at":"2015-10-19T22:34:44Z","updated_at":"2015-10-20T06:07:27Z","closed_at":null,"body":"When a node that is the current leader leaves the cluster it will do so without properly updating the cluster configuration. This happens because the state transition from LEADER to LEAVE prevents committing the updating cluster configuration."},"comment":{"url":"https://api.github.com/repos/atomix/copycat/issues/comments/149443545","html_url":"https://github.com/atomix/copycat/issues/20#issuecomment-149443545","issue_url":"https://api.github.com/repos/atomix/copycat/issues/20","id":149443545,"user":{"login":"madjam","id":8388825,"avatar_url":"https://avatars.githubusercontent.com/u/8388825?v=3","gravatar_id":"","url":"https://api.github.com/users/madjam","html_url":"https://github.com/madjam","followers_url":"https://api.github.com/users/madjam/followers","following_url":"https://api.github.com/users/madjam/following{/other_user}","gists_url":"https://api.github.com/users/madjam/gists{/gist_id}","starred_url":"https://api.github.com/users/madjam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/madjam/subscriptions","organizations_url":"https://api.github.com/users/madjam/orgs","repos_url":"https://api.github.com/users/madjam/repos","events_url":"https://api.github.com/users/madjam/events{/privacy}","received_events_url":"https://api.github.com/users/madjam/received_events","type":"User","site_admin":false},"created_at":"2015-10-20T06:07:27Z","updated_at":"2015-10-20T06:07:27Z","body":"Fully agree that this is an issue with a lot of interesting scenarios.\r\n\r\nMy understanding of the Raft protocol's behavior around cluster configuration is that when the current leader wants to leave it can first commit the updated configuration and then step down. Here is the relevant snippet from the paper: *The second issue is that the cluster leader may not be part of the new configuration. In this case, the leader steps down (returns to follower state) once it has committed the Cnew log entry.*\r\n\r\nMy fear is that exiting no matter what after a timeout can result in a cluster that can no longer reach consensus. For example if we have a 2 node cluster and one node decides to leave then the remaining node cannot do anything useful. For a 2+ node cluster we will have a slightly improved tolerance for this behavior: in a 3 node cluster the remaining 2 can still do useful work. But the the moment another node decides to just leave we are in trouble.\r\n\r\nMy preference would be to block the close() until a node can successfully leave. At least that way we are safe. \r\nAnd additionally also support another entry point to cluster reconfiguration. This can be useful in a 3 node scenario where one node just crashed and died and the admin wants to reconfigure the cluster so that it now only has 2 nodes."}}