{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/fleet/issues/1023","labels_url":"https://api.github.com/repos/coreos/fleet/issues/1023/labels{/name}","comments_url":"https://api.github.com/repos/coreos/fleet/issues/1023/comments","events_url":"https://api.github.com/repos/coreos/fleet/issues/1023/events","html_url":"https://github.com/coreos/fleet/issues/1023","id":48272971,"number":1023,"title":"fleet rebalancing feature","user":{"login":"rimusz","id":5215395,"avatar_url":"https://avatars.githubusercontent.com/u/5215395?v=3","gravatar_id":"","url":"https://api.github.com/users/rimusz","html_url":"https://github.com/rimusz","followers_url":"https://api.github.com/users/rimusz/followers","following_url":"https://api.github.com/users/rimusz/following{/other_user}","gists_url":"https://api.github.com/users/rimusz/gists{/gist_id}","starred_url":"https://api.github.com/users/rimusz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rimusz/subscriptions","organizations_url":"https://api.github.com/users/rimusz/orgs","repos_url":"https://api.github.com/users/rimusz/repos","events_url":"https://api.github.com/users/rimusz/events{/privacy}","received_events_url":"https://api.github.com/users/rimusz/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/coreos/fleet/labels/discussion","name":"discussion","color":"c7def8"},{"url":"https://api.github.com/repos/coreos/fleet/labels/feature","name":"feature","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2014-11-10T15:02:02Z","updated_at":"2015-09-23T19:56:13Z","closed_at":null,"body":"As per our discussion on https://groups.google.com/forum/#!topic/coreos-user/2TMksA4tMfQ\r\n\r\nCurrently fleet nicely balances units around cluster machines only for:\r\n1) new fresh installed cluster \r\n2) adding new units to the current cluster\r\n\r\nThe problem is if one of node gets reboot e.g. an update or we get a new node in the cluster, units do not get rebalanced back around all the nodes.\r\n\r\nI would suggest to have a simple rebalancing  for the fleet.\r\n1) Rebooted/new node pings fleet - I'm here give me some job :) \r\n2) fleet calculates all units which can be rebalanced, counts them, checks cluster nodes number and \r\nrandomly picks units and rebalances them. (the same procedure you use for deploying new units should be fine there)\r\n\r\ne.g.\r\n3 x nodes cluster with 20 units which can work on any node.\r\n1) 1 node gets rebooted for the upgrade.\r\n2) 20/3=6.66 round it to 7\r\n3) random 7 units get rebalanced on the rebooted/new node.\r\n\r\nCPU, RAM usage and etc should not be counted in, we do not want to complicated things too much :)\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/coreos/fleet/issues/comments/142711019","html_url":"https://github.com/coreos/fleet/issues/1023#issuecomment-142711019","issue_url":"https://api.github.com/repos/coreos/fleet/issues/1023","id":142711019,"user":{"login":"tjanczuk","id":822369,"avatar_url":"https://avatars.githubusercontent.com/u/822369?v=3","gravatar_id":"","url":"https://api.github.com/users/tjanczuk","html_url":"https://github.com/tjanczuk","followers_url":"https://api.github.com/users/tjanczuk/followers","following_url":"https://api.github.com/users/tjanczuk/following{/other_user}","gists_url":"https://api.github.com/users/tjanczuk/gists{/gist_id}","starred_url":"https://api.github.com/users/tjanczuk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tjanczuk/subscriptions","organizations_url":"https://api.github.com/users/tjanczuk/orgs","repos_url":"https://api.github.com/users/tjanczuk/repos","events_url":"https://api.github.com/users/tjanczuk/events{/privacy}","received_events_url":"https://api.github.com/users/tjanczuk/received_events","type":"User","site_admin":false},"created_at":"2015-09-23T19:56:13Z","updated_at":"2015-09-23T19:56:13Z","body":"For people who have a similar scenario to mine, here is how I worked around this limitation of fleet:\r\n\r\n1. I created a template unit file `my_service@.service` with `Global=true` setting\r\n2. I spawn X instances of this unit in the cluster, say 10. Given a 3 VM cluster, this results in 30 units running across the cluster. \r\n3. In case of a single VM failure, the VM just takes the instances that were running on it down with it. Fleet *does not* reschedule them on other VMs - but this is something I can live with given my requirements. \r\n4. When a new VM joins or re-joins the cluster, the `Global=true` setting in the unit template file will automatically have 10 instances of my_service started on that VM. \r\n"}}