{"action":"closed","issue":{"url":"https://api.github.com/repos/crate/crate/issues/1816","labels_url":"https://api.github.com/repos/crate/crate/issues/1816/labels{/name}","comments_url":"https://api.github.com/repos/crate/crate/issues/1816/comments","events_url":"https://api.github.com/repos/crate/crate/issues/1816/events","html_url":"https://github.com/crate/crate/issues/1816","id":63241335,"number":1816,"title":"[v. 0.45.6] Node dropping out of the cluster","user":{"login":"l7s","id":5430068,"avatar_url":"https://avatars.githubusercontent.com/u/5430068?v=3","gravatar_id":"","url":"https://api.github.com/users/l7s","html_url":"https://github.com/l7s","followers_url":"https://api.github.com/users/l7s/followers","following_url":"https://api.github.com/users/l7s/following{/other_user}","gists_url":"https://api.github.com/users/l7s/gists{/gist_id}","starred_url":"https://api.github.com/users/l7s/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/l7s/subscriptions","organizations_url":"https://api.github.com/users/l7s/orgs","repos_url":"https://api.github.com/users/l7s/repos","events_url":"https://api.github.com/users/l7s/events{/privacy}","received_events_url":"https://api.github.com/users/l7s/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-03-20T14:45:28Z","updated_at":"2015-03-20T21:46:34Z","closed_at":"2015-03-20T21:46:34Z","body":"Hi, I am running v. 0.45.6 and got the following problem:\r\n\r\n1. Node fails to ping master (note: retry 1 out of 3)\r\n```[2015-03-19 01:35:57,336][TRACE][discovery.zen.fd         ] [crate-eu-3] [master] failed to ping [[crate-eu-1][Nd1t6zk9TuGPzwDPjY3RAA][uk32.lan][inet[/192.168.10.32:4300]]{http_address=http://192.168.10.32:4200}], retry [1] out of [3]\r\norg.elasticsearch.transport.ReceiveTimeoutTransportException: [crate-eu-1][inet[/192.168.10.32:4300]][discovery/zen/fd/masterPing] request_id [5136942] timed out after [30000ms]\r\n        at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:356)\r\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\r\n        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\r\n        at java.lang.Thread.run(Thread.java:745)```\r\n\r\n2. Second later (with no further re-tries - should be 2 more according to configuration) it drops out:\r\n```[2015-03-19 01:35:57,749][WARN ][transport                ] [crate-eu-3] Received response for a request that has timed out, sent [30413ms] ago, timed out [413ms] ago, action [discovery/zen/fd/masterPing], node [[crate-eu-1][Nd1t6zk9TuGPzwDPjY3RAA][uk32.lan][inet[/192.168.10.32:4300]]{http_address=http://192.168.10.32:4200}], id [5136942]\r\n[2015-03-19 01:35:58,606][INFO ][io.crate.http.netty.HttpBlobHandler] channel closed: java.nio.channels.ClosedChannelException\r\n[2015-03-19 01:35:58,607][INFO ][io.crate.http.netty.HttpBlobHandler] channel closed: java.nio.channels.ClosedChannelException\r\n[2015-03-19 01:35:58,608][INFO ][io.crate.http.netty.HttpBlobHandler] channel closed: java.nio.channels.ClosedChannelException\r\n[2015-03-19 01:35:58,609][INFO ][io.crate.http.netty.HttpBlobHandler] channel closed: java.nio.channels.ClosedChannelException\r\n[2015-03-19 01:38:35,002][INFO ][monitor.jvm              ] [crate-eu-3] [gc][young][1884759][119710] duration [881ms], collections [1]/[1.3s], total [881ms]/[17m], memory [464.1mb]->[310.2mb]/[643.8mb], all_pools {[young] [151.9mb]->[1.7mb]/[177mb]}{[survivor] [22.1mb]->[16.5mb]/[22.1mb]}{[old] [290.1mb]->[291.9mb]/[444.6mb]}\r\n[2015-03-19 02:55:58,422][WARN ][monitor.jvm              ] [crate-eu-3] [gc][young][1889400][119821] duration [1.1s], collections [1]/[1.5s], total [1.1s]/[17.1m], memory [500.9mb]->[331.1mb]/[643.8mb], all_pools {[young] [176.8mb]->[6.3mb]/[177mb]}{[survivor] [4.7mb]->[4.8mb]/[22.1mb]}{[old] [319.4mb]->[320mb]/[444.6mb]}\r\n[2015-03-19 03:20:59,768][INFO ][monitor.jvm              ] [crate-eu-3] [gc][young][1890901][119844] duration [818ms], collections [1]/[1s], total [818ms]/[17.1m], memory [504mb]->[372.8mb]/[643.8mb], all_pools {[young] [171.8mb]->[40.1mb]/[177mb]}{[survivor] [5.6mb]->[5.1mb]/[22.1mb]}{[old] [326.5mb]->[327.5mb]/[444.6mb]}```\r\n\r\nIs it likely this has been fixed in newer releases? Is it worth upgrading to solve this? Or is there any other solution?\r\n\r\nRegards,\r\nChris\r\n\r\n<bountysource-plugin>\r\n---\r\nWant to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/9800028-v-0-45-6-node-dropping-out-of-the-cluster?utm_campaign=plugin&utm_content=tracker%2F505659&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F505659&utm_medium=issues&utm_source=github).\r\n</bountysource-plugin>"}}