{"action":"created","issue":{"url":"https://api.github.com/repos/rethinkdb/rethinkdb/issues/4934","labels_url":"https://api.github.com/repos/rethinkdb/rethinkdb/issues/4934/labels{/name}","comments_url":"https://api.github.com/repos/rethinkdb/rethinkdb/issues/4934/comments","events_url":"https://api.github.com/repos/rethinkdb/rethinkdb/issues/4934/events","html_url":"https://github.com/rethinkdb/rethinkdb/issues/4934","id":110574144,"number":4934,"title":"Whenever a new proxy is disconnected/connected, all other proxy seem to be disconnect","user":{"login":"mshi","id":1325507,"avatar_url":"https://avatars.githubusercontent.com/u/1325507?v=3","gravatar_id":"","url":"https://api.github.com/users/mshi","html_url":"https://github.com/mshi","followers_url":"https://api.github.com/users/mshi/followers","following_url":"https://api.github.com/users/mshi/following{/other_user}","gists_url":"https://api.github.com/users/mshi/gists{/gist_id}","starred_url":"https://api.github.com/users/mshi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mshi/subscriptions","organizations_url":"https://api.github.com/users/mshi/orgs","repos_url":"https://api.github.com/users/mshi/repos","events_url":"https://api.github.com/users/mshi/events{/privacy}","received_events_url":"https://api.github.com/users/mshi/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/rethinkdb/rethinkdb/labels/tp:bug","name":"tp:bug","color":"e102d8"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/rethinkdb/rethinkdb/milestones/104","html_url":"https://github.com/rethinkdb/rethinkdb/milestones/2.1.x","labels_url":"https://api.github.com/repos/rethinkdb/rethinkdb/milestones/104/labels","id":1249551,"number":104,"title":"2.1.x","description":null,"creator":{"login":"danielmewes","id":505365,"avatar_url":"https://avatars.githubusercontent.com/u/505365?v=3","gravatar_id":"","url":"https://api.github.com/users/danielmewes","html_url":"https://github.com/danielmewes","followers_url":"https://api.github.com/users/danielmewes/followers","following_url":"https://api.github.com/users/danielmewes/following{/other_user}","gists_url":"https://api.github.com/users/danielmewes/gists{/gist_id}","starred_url":"https://api.github.com/users/danielmewes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danielmewes/subscriptions","organizations_url":"https://api.github.com/users/danielmewes/orgs","repos_url":"https://api.github.com/users/danielmewes/repos","events_url":"https://api.github.com/users/danielmewes/events{/privacy}","received_events_url":"https://api.github.com/users/danielmewes/received_events","type":"User","site_admin":false},"open_issues":27,"closed_issues":2,"state":"open","created_at":"2015-08-11T07:18:11Z","updated_at":"2015-10-09T18:13:11Z","due_on":null,"closed_at":null},"comments":7,"created_at":"2015-10-09T01:52:05Z","updated_at":"2015-10-09T21:39:55Z","closed_at":null,"body":"Currently I have 4 servers clustered together. Each server instance that needs to access the database has a rethinkdb proxy service running. I have recently noticed that when one of the proxy server has a problem, and then I restart the service, it takes down all of the other proxy servers on _another_ instance with them. As a result, all the servers that are running the proxy will return errors along the lines of primary replica not available.\r\n\r\nThis is what I am starting the proxy with: `rethinkdb proxy --daemon --runuser rethinkdb --rungroup rethinkdb --pid-file /var/run/rethinkdb/instance1/proxy.pid --join 10.240.0.5:29015 -o 1 --log-file /var/lib/rethinkdb/instance1/data/log_file` And this is triggered in the init.d startup.\r\n\r\nI am running 2.1.4 currently."},"comment":{"url":"https://api.github.com/repos/rethinkdb/rethinkdb/issues/comments/146991875","html_url":"https://github.com/rethinkdb/rethinkdb/issues/4934#issuecomment-146991875","issue_url":"https://api.github.com/repos/rethinkdb/rethinkdb/issues/4934","id":146991875,"user":{"login":"mshi","id":1325507,"avatar_url":"https://avatars.githubusercontent.com/u/1325507?v=3","gravatar_id":"","url":"https://api.github.com/users/mshi","html_url":"https://github.com/mshi","followers_url":"https://api.github.com/users/mshi/followers","following_url":"https://api.github.com/users/mshi/following{/other_user}","gists_url":"https://api.github.com/users/mshi/gists{/gist_id}","starred_url":"https://api.github.com/users/mshi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mshi/subscriptions","organizations_url":"https://api.github.com/users/mshi/orgs","repos_url":"https://api.github.com/users/mshi/repos","events_url":"https://api.github.com/users/mshi/events{/privacy}","received_events_url":"https://api.github.com/users/mshi/received_events","type":"User","site_admin":false},"created_at":"2015-10-09T21:39:55Z","updated_at":"2015-10-09T21:39:55Z","body":"I'm running all of these on compute engine and I've set every canonical address to their local address for each config that host the database. And all internal IPs have all ports unblocked, so I don't think that's the case. I see them all connected in the admin panel with no issues reported. How long does the failover recovery take approximately?\r\n\r\nWould the appropriate approach be to catch the error and then continuously retry or is there a better way of handling this?\r\n\r\nI will try to set the canonical address of the proxy instances and give it a try later today during non-peak hours.\r\n\r\nThanks!"}}