{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/6082","labels_url":"https://api.github.com/repos/apache/spark/issues/6082/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/6082/comments","events_url":"https://api.github.com/repos/apache/spark/issues/6082/events","html_url":"https://github.com/apache/spark/pull/6082","id":75600180,"number":6082,"title":"[SPARK-7533] [YARN] Decrease spacing between AM-RM heartbeats.","user":{"login":"ehnalis","id":8371789,"avatar_url":"https://avatars.githubusercontent.com/u/8371789?v=3","gravatar_id":"","url":"https://api.github.com/users/ehnalis","html_url":"https://github.com/ehnalis","followers_url":"https://api.github.com/users/ehnalis/followers","following_url":"https://api.github.com/users/ehnalis/following{/other_user}","gists_url":"https://api.github.com/users/ehnalis/gists{/gist_id}","starred_url":"https://api.github.com/users/ehnalis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ehnalis/subscriptions","organizations_url":"https://api.github.com/users/ehnalis/orgs","repos_url":"https://api.github.com/users/ehnalis/repos","events_url":"https://api.github.com/users/ehnalis/events{/privacy}","received_events_url":"https://api.github.com/users/ehnalis/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":54,"created_at":"2015-05-12T13:23:18Z","updated_at":"2015-05-18T20:18:29Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/6082","html_url":"https://github.com/apache/spark/pull/6082","diff_url":"https://github.com/apache/spark/pull/6082.diff","patch_url":"https://github.com/apache/spark/pull/6082.patch"},"body":"Added faster RM-heartbeats on pending container allocations with multiplicative back-off.\r\nAlso updated related documentations."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/103197630","html_url":"https://github.com/apache/spark/pull/6082#issuecomment-103197630","issue_url":"https://api.github.com/repos/apache/spark/issues/6082","id":103197630,"user":{"login":"ehnalis","id":8371789,"avatar_url":"https://avatars.githubusercontent.com/u/8371789?v=3","gravatar_id":"","url":"https://api.github.com/users/ehnalis","html_url":"https://github.com/ehnalis","followers_url":"https://api.github.com/users/ehnalis/followers","following_url":"https://api.github.com/users/ehnalis/following{/other_user}","gists_url":"https://api.github.com/users/ehnalis/gists{/gist_id}","starred_url":"https://api.github.com/users/ehnalis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ehnalis/subscriptions","organizations_url":"https://api.github.com/users/ehnalis/orgs","repos_url":"https://api.github.com/users/ehnalis/repos","events_url":"https://api.github.com/users/ehnalis/events{/privacy}","received_events_url":"https://api.github.com/users/ehnalis/received_events","type":"User","site_admin":false},"created_at":"2015-05-18T20:18:29Z","updated_at":"2015-05-18T20:18:29Z","body":"I think so too, putting this in makes no harm, but can fasten the start-up of Spark jobs on not-so-contested clusters. I think we have two questions here:\r\n\r\n1) How much improvement do I get when there are enough resources when my AM wakes up?\r\nThis is very simple to answer and there's no need for evaluation. Sleeping for 200ms is 4800ms better than 5000ms. Any evaluation would be incorrect if it does not reports 4800ms difference.\r\n\r\n2) How much additional stress do we introduce to the RM by eagerly heartbeating against pending allocations on a loaded cluster?\r\nDepends on the characteristics of the cluster. I guess administrators would like to tailor and refine the eager allocation interval based on average load of RM. It's still okay to set it to 2.5 s, so it will jump into 5 s on the first \"allocation-miss\".\r\n\r\nSo everything can be a reality, but that is why we have these intervals configurable.\r\nTrivially, the dumb version will not adapt to the cluster's load, therefore it would need to be updated manually when load changes on RM.\r\n\r\n@tgravescs You think that 3 seconds of heartbeat would not be much load on the RM, but on some cluster it would! In same cases you might would like to HB on each 10th second, but eagerly HB with 5-second intervals! It depends on your cluster, but at least you have some adaptive mechanism, which will fasten start-up of Spark jobs, but will also consider the load of the RM."}}