{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/12853","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/12853/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/12853/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/12853/events","html_url":"https://github.com/elastic/elasticsearch/pull/12853","id":100737009,"number":12853,"title":"[qa] multinode tests fails when you run low on disk space (85%)","user":{"login":"dadoonet","id":274222,"avatar_url":"https://avatars.githubusercontent.com/u/274222?v=3","gravatar_id":"","url":"https://api.github.com/users/dadoonet","html_url":"https://github.com/dadoonet","followers_url":"https://api.github.com/users/dadoonet/followers","following_url":"https://api.github.com/users/dadoonet/following{/other_user}","gists_url":"https://api.github.com/users/dadoonet/gists{/gist_id}","starred_url":"https://api.github.com/users/dadoonet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dadoonet/subscriptions","organizations_url":"https://api.github.com/users/dadoonet/orgs","repos_url":"https://api.github.com/users/dadoonet/repos","events_url":"https://api.github.com/users/dadoonet/events{/privacy}","received_events_url":"https://api.github.com/users/dadoonet/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/build","name":"build","color":"5319e7"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/review","name":"review","color":"0052cc"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/test","name":"test","color":"5319e7"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/v2.0.0","name":"v2.0.0","color":"dddddd"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-08-13T10:48:11Z","updated_at":"2015-08-13T13:50:55Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/elastic/elasticsearch/pulls/12853","html_url":"https://github.com/elastic/elasticsearch/pull/12853","diff_url":"https://github.com/elastic/elasticsearch/pull/12853.diff","patch_url":"https://github.com/elastic/elasticsearch/pull/12853.patch"},"body":"Indeed, we check within the test suite that we have not unassigned shards.\r\n\r\nBut when the test starts on my machine I get:\r\n\r\n```\r\n[elasticsearch] [2015-08-13 12:03:18,801][INFO ][org.elasticsearch.cluster.routing.allocation.decider] [Kehl of Tauran] low disk watermark [85%] exceeded on [eLujVjWAQ8OHdhscmaf0AQ][Jackhammer] free: 59.8gb[12.8%], replicas will not be assigned to this node\r\n```\r\n\r\n```\r\n  2> REPRODUCE WITH: mvn verify -Pdev -Dskip.unit.tests -Dtests.seed=2AE3A3B7B13CE3D6 -Dtests.class=org.elasticsearch.smoketest.SmokeTestMultiIT -Dtests.method=\"test {yaml=smoke_test_multinode/10_basic/cluster health basic test, one index}\" -Des.logger.level=ERROR -Dtests.assertion.disabled=false -Dtests.security.manager=true -Dtests.heap.size=512m -Dtests.locale=ar_YE -Dtests.timezone=Asia/Hong_Kong -Dtests.rest.suite=smoke_test_multinode\r\nFAILURE 38.5s | SmokeTestMultiIT.test {yaml=smoke_test_multinode/10_basic/cluster health basic test, one index} <<<\r\n   > Throwable #1: java.lang.AssertionError: expected [2xx] status code but api [cluster.health] returned [408 Request Timeout] [{\"cluster_name\":\"prepare_release\",\"status\":\"yellow\",\"timed_out\":true,\"number_of_nodes\":2,\"number_of_data_nodes\":2,\"active_primary_shards\":3,\"active_shards\":3,\"relocating_shards\":0,\"initializing_shards\":0,\"unassigned_shards\":3,\"delayed_unassigned_shards\":0,\"number_of_pending_tasks\":0,\"number_of_in_flight_fetch\":0,\"task_max_waiting_in_queue_millis\":0,\"active_shards_percent_as_number\":50.0}]\r\n```\r\n\r\nI propose here to define for all integration tests:\r\n\r\n* `cluster.routing.allocation.disk.watermark.low:200mb`\r\n* `cluster.routing.allocation.disk.watermark.high:100mb`\r\n\r\nCloses #12852."},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/130679612","html_url":"https://github.com/elastic/elasticsearch/pull/12853#issuecomment-130679612","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/12853","id":130679612,"user":{"login":"dakrone","id":19060,"avatar_url":"https://avatars.githubusercontent.com/u/19060?v=3","gravatar_id":"","url":"https://api.github.com/users/dakrone","html_url":"https://github.com/dakrone","followers_url":"https://api.github.com/users/dakrone/followers","following_url":"https://api.github.com/users/dakrone/following{/other_user}","gists_url":"https://api.github.com/users/dakrone/gists{/gist_id}","starred_url":"https://api.github.com/users/dakrone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dakrone/subscriptions","organizations_url":"https://api.github.com/users/dakrone/orgs","repos_url":"https://api.github.com/users/dakrone/repos","events_url":"https://api.github.com/users/dakrone/events{/privacy}","received_events_url":"https://api.github.com/users/dakrone/received_events","type":"User","site_admin":false},"created_at":"2015-08-13T13:50:55Z","updated_at":"2015-08-13T13:50:55Z","body":"> Sounds like elasticsearch has bad defaults to me.\r\n\r\n@rmuir explain why you think these are bad? I could see changing to 90/95% perhaps, but there isn't a good way to support the wide array of disk sizes people run ES on other than relative values.\r\n\r\n> Should we change Elasticsearch defaults to absolute values?\r\n\r\nNo. I think relative values are the only way to support as many disk sizes as we can by default. And if not, that's why they are dynamically configurable."}}