{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/7532","labels_url":"https://api.github.com/repos/apache/spark/issues/7532/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/7532/comments","events_url":"https://api.github.com/repos/apache/spark/issues/7532/events","html_url":"https://github.com/apache/spark/pull/7532","id":96010597,"number":7532,"title":"[SPARK-4751] Dynamic allocation in standalone mode","user":{"login":"andrewor14","id":2133137,"avatar_url":"https://avatars.githubusercontent.com/u/2133137?v=3","gravatar_id":"","url":"https://api.github.com/users/andrewor14","html_url":"https://github.com/andrewor14","followers_url":"https://api.github.com/users/andrewor14/followers","following_url":"https://api.github.com/users/andrewor14/following{/other_user}","gists_url":"https://api.github.com/users/andrewor14/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewor14/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewor14/subscriptions","organizations_url":"https://api.github.com/users/andrewor14/orgs","repos_url":"https://api.github.com/users/andrewor14/repos","events_url":"https://api.github.com/users/andrewor14/events{/privacy}","received_events_url":"https://api.github.com/users/andrewor14/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":60,"created_at":"2015-07-20T08:12:06Z","updated_at":"2015-07-31T21:31:27Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/7532","html_url":"https://github.com/apache/spark/pull/7532","diff_url":"https://github.com/apache/spark/pull/7532.diff","patch_url":"https://github.com/apache/spark/pull/7532.patch"},"body":"Dynamic allocation is a feature that allows a Spark application to scale the number of executors up and down dynamically based on the workload. Support was first introduced in YARN since 1.2, and then extended to Mesos coarse-grained mode recently. Today, it is finally supported in standalone mode as well!\r\n\r\nI tested this locally and it works as expected. This is WIP because unit tests are coming."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/126816568","html_url":"https://github.com/apache/spark/pull/7532#issuecomment-126816568","issue_url":"https://api.github.com/repos/apache/spark/issues/7532","id":126816568,"user":{"login":"andrewor14","id":2133137,"avatar_url":"https://avatars.githubusercontent.com/u/2133137?v=3","gravatar_id":"","url":"https://api.github.com/users/andrewor14","html_url":"https://github.com/andrewor14","followers_url":"https://api.github.com/users/andrewor14/followers","following_url":"https://api.github.com/users/andrewor14/following{/other_user}","gists_url":"https://api.github.com/users/andrewor14/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewor14/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewor14/subscriptions","organizations_url":"https://api.github.com/users/andrewor14/orgs","repos_url":"https://api.github.com/users/andrewor14/repos","events_url":"https://api.github.com/users/andrewor14/events{/privacy}","received_events_url":"https://api.github.com/users/andrewor14/received_events","type":"User","site_admin":false},"created_at":"2015-07-31T21:31:27Z","updated_at":"2015-07-31T21:31:27Z","body":"@vanzin I just looked at the code again and there doesn't seem to be a great way to separate it out. This is because, in the \"one executor per worker\" case, we actually need to keep scheduling cores on the existing executors even when we're already at the limit. E.g. if the limit is 3 and we have already scheduled 1 core on each worker (1, 1, 1), we should keep scheduling even though we already \"have\" 3 executors.\r\n\r\nInstead, I updated the comment and variable placement to make this easier to read."}}