{"action":"created","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/10242","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/10242/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/10242/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/10242/events","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/10242","id":90456425,"number":10242,"title":"Scheduler can't compare pods across namespaces with/without limits","user":{"login":"bprashanth","id":10927820,"avatar_url":"https://avatars.githubusercontent.com/u/10927820?v=3","gravatar_id":"","url":"https://api.github.com/users/bprashanth","html_url":"https://github.com/bprashanth","followers_url":"https://api.github.com/users/bprashanth/followers","following_url":"https://api.github.com/users/bprashanth/following{/other_user}","gists_url":"https://api.github.com/users/bprashanth/gists{/gist_id}","starred_url":"https://api.github.com/users/bprashanth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bprashanth/subscriptions","organizations_url":"https://api.github.com/users/bprashanth/orgs","repos_url":"https://api.github.com/users/bprashanth/repos","events_url":"https://api.github.com/users/bprashanth/events{/privacy}","received_events_url":"https://api.github.com/users/bprashanth/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/priority/P1","name":"priority/P1","color":"eb6420"},{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/team/master","name":"team/master","color":"d2b48c"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/milestones/12","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/milestones/v1.0-candidate","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/milestones/12/labels","id":1086011,"number":12,"title":"v1.0-candidate","description":"These are candidates for inclusion to v1.0 but will be pushed to v1.0-bubble if they're triaged as not meeting the v1.0 bar as a required launch blocker.","creator":{"login":"alex-mohr","id":8729097,"avatar_url":"https://avatars.githubusercontent.com/u/8729097?v=3","gravatar_id":"","url":"https://api.github.com/users/alex-mohr","html_url":"https://github.com/alex-mohr","followers_url":"https://api.github.com/users/alex-mohr/followers","following_url":"https://api.github.com/users/alex-mohr/following{/other_user}","gists_url":"https://api.github.com/users/alex-mohr/gists{/gist_id}","starred_url":"https://api.github.com/users/alex-mohr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alex-mohr/subscriptions","organizations_url":"https://api.github.com/users/alex-mohr/orgs","repos_url":"https://api.github.com/users/alex-mohr/repos","events_url":"https://api.github.com/users/alex-mohr/events{/privacy}","received_events_url":"https://api.github.com/users/alex-mohr/received_events","type":"User","site_admin":false},"open_issues":17,"closed_issues":74,"state":"open","created_at":"2015-04-27T19:03:36Z","updated_at":"2015-06-25T23:05:33Z","due_on":null,"closed_at":null},"comments":20,"created_at":"2015-06-23T18:23:41Z","updated_at":"2015-06-26T14:26:35Z","closed_at":null,"body":"Setup a 3 node cluster, it'll look like:\r\n```\r\nNAME                                               READY     REASON    RESTARTS   AGE\r\nelasticsearch-logging-v1-trk5e                     1/1       Running   0          2m\r\nelasticsearch-logging-v1-u0pcb                     1/1       Running   0          2m\r\nfluentd-elasticsearch-e2e-test-beeps-minion-12g9   1/1       Running   0          1m\r\nfluentd-elasticsearch-e2e-test-beeps-minion-5bqu   1/1       Running   0          1m\r\nfluentd-elasticsearch-e2e-test-beeps-minion-jir8   1/1       Running   0          1m\r\nkibana-logging-v1-fgfdf                            1/1       Running   0          2m\r\nkube-dns-v3-n17tg                                  3/3       Running   0          2m\r\nmonitoring-heapster-v4-plzg4                       1/1       Running   0          2m\r\nmonitoring-influx-grafana-v1-hpic0                 2/2       Running   0          2m\r\n```\r\n3 nodes, 9 containers, in theory we should end up with 3 per node but we rarely do because the node scores are affected by the rate of creation of the fluentd pods. \r\n\r\nCreate an rc with 100 pods in a custom namespace and you'll end up with all 100 on the same node. This is because the occupancy score of the one node never catches up to the other 2, because the cluster addons have container cpu and memory populated via the default limit range while the 100 pod rc does not. "},"comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/comments/115707405","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/10242#issuecomment-115707405","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/10242","id":115707405,"user":{"login":"gmarek","id":1703842,"avatar_url":"https://avatars.githubusercontent.com/u/1703842?v=3","gravatar_id":"","url":"https://api.github.com/users/gmarek","html_url":"https://github.com/gmarek","followers_url":"https://api.github.com/users/gmarek/followers","following_url":"https://api.github.com/users/gmarek/following{/other_user}","gists_url":"https://api.github.com/users/gmarek/gists{/gist_id}","starred_url":"https://api.github.com/users/gmarek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gmarek/subscriptions","organizations_url":"https://api.github.com/users/gmarek/orgs","repos_url":"https://api.github.com/users/gmarek/repos","events_url":"https://api.github.com/users/gmarek/events{/privacy}","received_events_url":"https://api.github.com/users/gmarek/received_events","type":"User","site_admin":false},"created_at":"2015-06-26T14:26:35Z","updated_at":"2015-06-26T14:26:35Z","body":"I see that I had missed quite a lot by going to conferences, as it took me some time to understand what's the issue.\r\n\r\nIIUC we have a problem because current scheduling algorithm can assign \"all new pods\" to the single machine if it turns out that it's the one with slightly more free resources than the rest. I'm not quite sure what's the role of the namespace in this picture. Is it only LimitRange related, i.e. sets default limit on resources for Pods?\r\n\r\nMy 2c is that 0 resources is not very different from 1m resources, so I don't think that requiring \"non-zero\" requests will solve anything (except \"forgot to declare\" cases, but that won't be all of them).\r\n\r\nEither way I think that the best way to go, at least for now, is to go with @bgrant0607 point (2) / refined @bprashanth PR #10243. E.g. we can lower the score of nodes which are running multiple pods with the same set of labels (kind of @bgrant0607 point (3), but on the scoring, not spreading side of the algorithm). Or is there a reason why we don't want to influence score in this way?\r\n\r\nAs for why @daviopp brought me here: we can easily change `max_pods` to `max_pods_per_sth` to make it scale with something, e.g. number of cores on the machine. IIRC LimitRange goal was to, in a sense, limit the number of pods/resources used per node per user (namespace). I agree with @davidopp that for default namespace is pretty much the same, as limiting the number of pods per node, so there's no point of having LimitRange for it (except consistency)."}}